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ABSTRACT 

This  study  develops  a  computer  assisted  instruction 
response  evaluation  technique  to  aid  CAI  lesson  writers  in 
course  development.   The  objectives  for  this  technique  was 
to  experimentally  identify  principal  incorrect  response 
forms  and  determine  characteristic  error  patterns  for  a 
given  concept  to  be  taught.   Using  a  specific  concept,  the 
proposed  technique  was  evaluated.   Sixty-one  subjects  were 
randomly  selected  from  three  different  universities  and 
assigned  to  solve  a  problem-solving  task.   Using  mental 
imagery  and  oral  reporting,  they  produced  the  necessary 
protocol  for  this  experiment.   Although  the  response  eval- 
uation technique  developed  in  this  study  was  reasonably 
successful,  a  specific  response  error  pattern  for  each 
subject  could  not  be  determined. 
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I.   INTRODUCTION 

Abrams  (1971)  asserts  that  the  most  difficult  problems 
impeding  the  progress  of  computer-assisted  instruction  (CAI) 
are  those  involving  the  basics  of  the  educational  process 
itself  and  the  inherent  complexity  of  CAI  from  the  point  of 
view  of  the  educators.   While  a  good  CAI  system  will  not 
appear  complex  to  the  students  and  the  instructors  who  use 
it,  the  existing  and  proposed  new  system  are  difficult  for 
even  knowledgeable  educators  to  evaluate  for  applicability 
and  desirability. 

Even  more  basic  is  the  problem  of  the  use  of  diverse 
forms  of  responses  to  indicate  the  attainment  of  knowledge 
and  understanding.   This  problem  is  associated  with  two 
questions : 

(1)  What  type  of  response  is  required  from  the 
student  and  the  CAI  system  for  a  given 
educational  evolution? 

(2)  Will  this  response  be  a  true  indicator  of 
the  student's  understanding  of  the  concepts 
involved? 

These  are  difficult,  nevertheless,  important  questions  as 
indicated  by  Simon:   "the  possibilities  that  are  opened  up 
for  improving  human  problem-solving,  thinking,  and  decision- 
making activities"  are  "at  least  as  important"  as  the 
genuine  revolutionary  "developments  [which  are  now  apparent] 
in  automation."   Jweissinger-Baylon,  1978,  p.  ij  . 


The  author  shares  both  Abram's  concern  and  Simon's  con- 
fidence for  this  intrinsic  problem  area  of  computer-assisted 
instruction.   Although  much  has  been  accomplished  in  this 
problematical  area,  much  remains  to  be  solved.   Until  the 
many  factors  involved  in  the  response  evaluation  process  is 
more  fully  explored  and  more  thoroughly  understood,  it  is 
the  author's  opinion  that  it  will  be  difficult  for  CAI  to 
realize  its  full  potential. 

There  have  been  many  variations  in  the  definition  of 

computer-assisted  instruction  (CAI)  in  the  past  two  decades. 

The  one  definition  that  encompasses  the  whole  field  and 

which  will  be  used  in  this  paper  is  best  defined  by  Abrams : 

.  .  .  the  use  of  computers  to  aid  teachers  and  students  in 
the  educational  process,  utilizing  such  functions  as  pre- 
senting materials  and  problems,  guiding  a  student's  path 
through  a  course  by  selecting  the  material  to  be  pre- 
sented or  by  assigning  tasks  to  be  completed  away  from  the 
computer,  or  any  combination  of  these  functions. 

[Abrams,  19  71,  p.  4j 
Therefore,  CAI  does  not  just  simply  imply  typing  out 
questions  to  which  students  respond  and  by  then  giving  cues 
and  different  hits,  CAI  includes  the  computational  use  of 
the  computer,  when  appropriate,  as  part  of  the  instruction. 
Computer-assisted  instruction  systems  may  be  generally 
viewed  in  two  ways:  machine-directed  and  student-directed. 
In  the  machine-directed  approach,  various  paths  and  alterna- 
tives through  instructional  material  is  programmed  into  a 
machine.   In  this  approach  there  must  be  prior  knowledge  of 
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the  needs  of  the  student  and  a  predetermined  method  by  which 
the  computer  can  evaluate  the  student's  current  state  of 
knowledge.   In  addition,  the  student's  past  performance 
record  and  techniques  for  describing  a  further  path  for  each 
student,  which  would  optimize  his  learning  experience,  must 
be  at  the  machine's  disposal. 

In  the  student-directed  approach,  the  sequence  of  the 
material  presented  is  altered  only  at  the  request  of  the 
student.   The  approach  uses  the  aspect  of  learning  by  dis- 
covery and  experimentation.   This  area,  however,  has  been 
almost  untouched  by  current  research  and,  therefore,  much 
less  has  been  done  to  utilize  this  type  of  approach. 

This  paper  deals  primarily  with  the  machine-directed  CAI 
approach.   A  new  computer-assisted  instruction  (CAI)  tech- 
nique with  the  potential  to  improve  the  effectiveness  of 
CAI  was  developed  and  evaluated.   This  technique  experi- 
mentally identifies  the  principal  incorrect  forms  of  any 
given  concept  that  is  taught.   Characteristic  error  patterns 
are  associated  with  incorrect  forms  so  that  the  subject  may 
be  presented  with  a  CAI  response  tailored  to  that  particular 
incorrect  concept  that  he  has  acquired.   New  instructional 
material  or  other  techniques  could  then  be  utilized  depend- 
ing upon  the  student's  understanding  or  lack  of  understand- 
ing of  specific  concepts. 

Absence  of  any  contributing  literature  in  the  area  of 
CAI  response  evaluation  and  personal  communications  with 
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prominent  individuals  in  the  area  of  CAI ,  indicate  that  the 
approach  discussed  in  this  study,  is  the  first  significant 
attempt  to  provide  a  specific  set  of  rules  or  guidelines 
for  evaluating  the  subject's  responses.   This  does  not  im- 
ply that  the  response  evaluation  area  of  CAI  was  never  con- 
sidered or  discussed,  but  that  a  set  of  rules  has  never 
been  formalized.   The  following  method  has  therefore  been 
designed  to  provide  CAI  lesson  writers  with  an  additional  tool 
for  structuring  CAI  course  material. 
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II.   BACKGROUND 

A.   THE  NEED  FOR  A  RESPONSE  EVALUATION  TECHNIQUE 

The  use  of  various  computer-assisted  instruction  systems 
has  been  on  the  upswing  in  the  last  decade.   This  growth  has 
been  desirable  and  is  a  result  of  the  increased  amount  of 
human  knowledge  that  needs  to  be  attained,  the  ever  increas- 
ing number  of  students  to  be  taught,  and  the  availability  of 
qualified  instructors.   Early  studies  revealed  that  consist- 
ent time  savings  occurred  during  a  three  year  experiment 
using  microcomputers  to  manage  students'  self-paced  pro- 
gression through  a  course  (Evans  and  Johns,  1979).   Orlansky 
and  String  (1979)  ,  in  an  iconoclastic  attack  on  the  cost- 
effectiveness  of  thirty  studies  of  military  computer-based 
education  (CBE)  have  also  concluded  that  a  thirty  percent 
time  savings  is  possible.   It  can  therefore  be  generally 
agreed  that  CAI  can  achieve  educational  objectives  more  effi- 
ciently and  more  effectively  than  other  teaching  methods. 
However,  four  basic  educational  factors  accentuate  the  need 
for  a  computer-assisted  instruction  (CAI)  response  evalua- 
tion technique. 

The  first  factor  is  the  trend  toward  individualized 
instruction.   The  fewer  the  number  of  students  for  which  a 
teacher  is  responsible,  the  more  time  and  individual 
attention  each  student  will  receive.   Therefore,  from  a 
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theoretical  aspect,  the  more  each  student  will  gain  from  his 
educational  experience.   Conventional  mass  media  such  as 
books,  films,  television,  and  even  programmed  instruction 
are  inherently  incapable  of  individualization.   Books,  films, 
and  television  programs  would  have  to  be  rewritten  to  fit 
each  student,  and  conventional  programmed  instruction  systems 
lack  the  flexibility  and  decision-making  capability  required 
of  a  truly  effective  individualized  system.   CAI  seems  to 
have  the  potential  to  meet  these  requirements.   However,  the 
current  relatively  sophisticated  computer-assisted  instruc- 
tion systems  are  only  tailored  towards  students  who  are 
homogeneous  in  background,  ability,  cognitive  style,  and 
motivation.   The  systems  are  incapable  of  individualization 
in  the  sense  of  structuring  a  response  as  a  true  indicator 
of  the  student's  understanding  of  the  concepts  involved. 
However,  by  evaluating  the  subject  response  and  then  pro- 
viding a  tailor-made  CAI  response  to  reflect  understanding 
or  lack  of  understanding  of  a  given  concept,  seems  to  have 
the  potential  to  improve  the  effectiveness  of  CAI  and  its 
value  to  individualized  instruction. 

The  second  factor  is  the  evaluation  of  the  student's 
learning  progress.   Although,  sophisticated  CAI  systems 
keep  track  of  the  number  of  wrong  responses  for  a  particu- 
lar concept,  they  do  not  identify  to  the  educator  the 
incorrect  concept  that  the  student  learns.   Using  a  response 
evaluation  technique,  the  wrong  concept  may  be  identified 
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and  new  instructional  material,  review  material,  drill  and 
practice  sessions,  tutoring,  or  other  techniques  could  then 
be  utilized  depending  on  the  student's  understanding. 

The  trend  towards  the  use  of  microcomputers  is  the  third 
factor.   Computer-assisted  instruction  has  typically  been 
applied  using  large,  dedicated  computer  systems  and  special 
CAI  languages.   There  are  virtually  no  limitation  on  memory 
or  file  sizes  for  the  instructional  material.   However, 
microcomputers  are  now  being  sold  at  prices  comparable  to 
color  television  sets  and  their  use  in  educational  settings 
will  skyrocket  in  the  next  few  years.   Invariably,  CAI  will 
be  one  of  the  uses  to  which  these  computer  systems  are  put. 
These  systems,  however,  have  inherent  limitations.   The  most 
important  being  memory  size.   Since,  these  limitations  are 
fixed  attributes  of  the  systems ,  they  must  be  dealt  with 
creatively.   One  method  is  to  use  a  response  evaluation 
technique.   Using  such  a  technique,  the  size  of  the  programs 
may  be  reduced  by  limiting  the  required  number  of  paths  or 
alternatives  through  the  instructional  material. 

Finally,  the  fourth  factor  is  the  limited  applicability 
of  computer-assisted  instruction  to  university  level  subjects. 
Weissinger-Baylon  asserts  that  according  to  Atkinson,  "the 
relatively  sophisticated  computer-assisted  instruction 
systems  that  have  been  successfully  introduced  for  reading 
and  elementary  areas  are  neither  applicable  to  under- 
graduate mathematics  nor  to  other  university  level  subjects; 

too  little  is  known  of  the  structure  of  the  required  skills 
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or  of  the  psychological  processes  for  their  mastery."   Until 
the  type  of  response  for  certain  concepts  are  more  fully 
explored  and  more  thoroughly  understood  it  will  be  difficult 
for  CAI  to  realize  its  full  potential. 

B.   RESEARCH  OBJECTIVES 

The  most  difficult  problem  impeding  the  progress  of  com- 
puter-assisted instruction  are  those  involving  the  basics  of 
the  educational  process  itself.   The  type  of  response  that 
must  be  obtained  from  the  student  to  get  an  indicator  of  his 
true  understanding  of  the  concept,  is  a  difficult  question. 
Although  there  has  been  some  work  performed  by  Newell,  Simon, 
Shaw,  and  Weissinger-Baylon  in  the  area  of  human  problem 
solving  (Weissinger-Baylon,  1978,  p.  s]  ,  it  has  only  been 
applied  to  the  area  of  computer-assisted  instruction  in  a 
very  limited  amount.   Substantial  progress  in  this  direction, 
however,  will  permit  far  reaching  new  approaches  to  education 

The  present  study  is  conceived  as  one  step  in  a  longer 
program,  the  development  of  major  improvements,  efficiency 
and  value  of  computer-assisted  instruction  systems.   The 
immediate  objective  is  to  develop  and  evaluate  a  new  com- 
puter-assisted instruction  (CAI)  technique  using  mental 
imagery  and  verbal  responses  to  experimentally  identify  the 
principal  incorrect  forms  of  a  given  concept  that  the  stu- 
dents usually  learns.   Characteristic  error  patterns  will  be 
identified  and  associated  with  incorrect  concept  forms  on  a 
diagnostic  test  and  will  be  evaluated  in  regards  to  its 
application  to  a  computer-assisted  instruction  system. 
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C.   TASK  SELECTION 

Task  selection  for  the  development  and  evaluation  of  any 
new  technique  or  procedure  must  be  based  on  many  factors  and 
therefore  makes  it  a  difficult  chore.   Weissinger-Baylon, 
from  his  research,  states  that  the  most  important  factors  in 
task  selection  are:   "emphasis  on  tractability  or  support 
for  the  experimenters  hypothesis  must  be  minimized;  the  re- 
search must  have  application,  at  least  potentially;  and 
finally,  the  complexity  of  the  task  must  not  limit  the 
communic ability  of  the  findings."   The  author  agrees  with 
Weissinger-Baylon 's  findings,  but  adds  that  a  task  may  be 
selected  because  of  it's  interest  to  the  analyst  and  because 
it  complements  the  work  of  others. 

The  choice  for  the  selection  of  mathematics  as  a  field 
to  test  and  evaluate  responses  for  a  CAI  system  was  made 
as  a  result  of  its  many  advantages:   "for  the  chosen  problem 
area,  both  concept  and  vocabulary  are  highly  developed;  ade- 
quate intellectual  resources  are  available  for  this  topic; 
its  method  and  administrative  procedures  are  convenient; 
and  it  is  an  important  area  of  application,  controlling  al- 
most all  entries  to  attractive  professions." 

Determining  a  problem  area  within  the  chosen  field  was 
the  next  stage  in  the  selection  process.   It  was  a  difficult 
task.   The  material  had  to  satisfy  both  the  requirements  of 
meeting  the  objectives  of  this  research  and,  most  impor- 
tantly, be  within  the  grasp  of  existing  information 
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processing  methodologies. 

Weissinger-Baylon's  personal  communication  with  Karl 
deLeeuw  (weissinger-Baylon,  1978,  p.  30|  ,  proposed  an 
analysis  of  a  group  of  closely  related  concepts  having  to 
do  with  "bounding"  of  sets.   It  was  felt  such  a  concept 
would  adequately  propose  or  illustrate,  at  least  theoreti- 
cally, the  technique  to  evaluate  and  analyze  responses  for 
a  CAI  system.   The  problem  that  arose,  however,  was  what 
type  of  bounding  concept  to  use — the  greatest  lower  bound, 
the  least  upper  bound,  or  determine  if  the  set  is  bounded 
above  or  below. 

Preliminary  studies  conducted  by  Weissinger-Baylon  on 
the  proposed  concepts,  suggested  that  the  focus  of  the 
analysis  for  a  response  evaluation  process  for  a  CAI 
system,  be  narrowed  to  a  single  definition.   His -studies 
on  the  upper  bound  concept  produced  the  most  interesting 
results  and  it  was  determined  that  the  importance  of  the 
upper  bound  concept,  its  manageability,  and  Weissinger- 
Baylon's  prior  research  in  this  area,  made  this  concept 
the  most  attractive  candidate  for  this  research  study  in 
evaluating  the  response  process  for  a  CAI  system. 

D.   METHODOLOGICAL  OVERVIEW 

The  development  of  a  suitable  methodology  proceeded  in 
stages.   Test  results  from  a  research  study  conducted  by 
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Weissinger-Baylon  on  the  upper-bound  of  sets  concept,  were 
made  available  for  this  research,  and  pre-empted  the  need 
for  duplicate  and  redundant  testing  in  the  initial  phase  of 
the  study. 

1.   Subjects 

The  sixty-one  students  who  served  as  subjects  were 
volunteers  from  three  different  universities:   the  Naval 
Postgraduate  School,  Stanford  University  and  Colorado  State 
University.   Subjects  were  selected  to  provide  a  reasonable 
balanced  representation  by  sex,  specialization  and  profes- 
sional levels.   They  were  obtained  from  bulletin  board 
announcements,  distribution  of  sign-up  sheets  and  by 
personal  request.   With  the  exception  of  the  Stanford  Uni- 
versity students,  who  received  a  $6.00  fee,  no  other 
compensation  was  offered  for  the  subject's  participation  in 
this  experiment.   The  mix  consisted  of  23%  females  and  77% 
males.   Of  these,  33%  were  graduate  or  undergraduate  stu- 
dents in  the  mathematical  sciences  (Computer  science. 
Statistics,  Operations  research  and  Mathematical  sciences) 
and  67%  were  undergraduates  or  non-mathematical  science 
graduates.   Prior  research  had  dictated  that  subjects  with- 
out university  mathematics  would  experience  difficulty  with 
this  particular  concept  and  were  therefore  required  to  have 
had  at  least  two  years  of  calculus. 


19 


2 .   Design 

The  problem-solving  task  was  initiated  by  looking 
at  fifteen  different,  but  related  definitions  from  advanced 
calculus.   From  the  results,  a  final  version  was  produced, 
which  focused  on  one  definition:   the  upper  bound  of  a  set 
of  real  numbers.   The  criteria  for  an  upper  bound  of  a  set 
Was  met  if  there  existed  a  real  number  which  was  greater 
than  every  element  of  the  set. 

Nine  problems  were  chosen  from  standard  texts  of  ad- 
vanced calculus  and  grouped  into  the  following  four  types 
of  sets : 

(1)  finite  sequences:  e.g.  (1,2,3,4) 

(2)  intervals:  e.g.  (a,b) 

(3)  convergent  infinite  sequences: 

e.g.  (l,  1/2,  1/3,  ...1/n,...} 

(4)  divergent  infinite  sequences: 

e.g.  il,2,3,...n,..vi 

[Weissinger-Baylon,  1978,  p.  32] 
In  addition,  the  above  sets  were  presented  in  the  following 
methods : 

(1)  enumerative:  {1,2,3,4} 

(2)  nominal:  N,  the  natural  numbers 

(3)  attributive:  finite  set 

[Weissinger-Baylon,  1978,  p.  32] 
The  sets  were  randomized  by  type   and  presentation  method. 
These  representations  vary  slightly  from  the  final  de- 
sign since  they  were  modified  to  make  them  suitable  for  a 
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wide  range  of  subjects.   Further,  it  is  well  to  note  that 
the  degree  of  accuracy  will  be  based  on  the  degree  to  which 
the  subject's  responses  match  the  existing  textbooks.   The 
production  of  imagery  or  step  by  step  analogy  of  the  prob- 
lem set  was  not  consciously  a  criterion. 

3 .   Procedure 


The  detailed,  step-by-step  procedure  for  the  CAI 
response  evaluation  technique  is  given  in  chapter  3 .   The 
procedural  model,  as  designed,  consists  of  seven  major 
components:   selection  of  target  content,  design  of 
problem-solving  task,  selection  of  protocol  methodology, 
pretesting,  refining  the  task,  administering  the  final 
version  of  the  task,  and  analysis  of  results. 

The  procedural  methodology  that  was  used  for  adminis- 
tering the  problem-solving  task  consisted  of  contacting 
the  subject  prior  to  the  experiment  and  agreeing  on  a  set 
time  for  administration  of  the  task.   Before  beginning  the 
problem-solving  task,  the  subjects  were  interviewed  to 
determine  their  qualifications  for  testing  on  the 
particular  concept.   In  addition,  during  the  interview  the 
subjects  provided  the  following  information:   year. 
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department,  and  educational  and  career  plans.   When  the 

participant  was  found  to  be  qualified,  he  was  given  the 

following  instructions : 

Instructions.  On  the  next  page,  you  will  find  a  defini- 
tion taken  from  an  undergraduate  mathematics  course.  At 
first  the  language  may  seem  imposing,  but  the  concept  is 
not  difficult.  Please  read  carefully,  and  try  to  under- 
stand it  as  well  as  you  can.  Next,  go  on  to  the  practice 
problem  and  the  questions  which  follow: 

We  would  like  you  to  think  out  loud  as  much  as  possible. 
If  you  use  any  mental  imagery  in  your  thinking,  please 
draw  or  describe  it.   At  the  end  of  each  problem,  you 
are  to  write  a  complete  summary  of  your  thought  process. 

You  may  begin  when  you  wish.   There  is  no  time  limit. 
Please  ask  any  questions  which  you  feel  are  necessary. 
Do  not  forget  to  think  out  loud,  or  draw  or  describe  any 
mental  imagery  you  may  have. 

[Weissinger-Baylon,  1978,  p.  3^ 

A  tape  recorder  was  then  put  in  front  of  the  subject  to 

report  his  oral  responses,  if  any  occurred.   When  the 

subject  indicated  that  he  understood  the  instructions,  he 

was  given  the  test  booklet  and  then  asked  to  start  the 

exam.   The  examiner  was  seated  beside  the  subject  to 

observe  and  take  notes.   At  the  end  of  the  entire  session, 

the  subject  was  again  interviewed  to  clarify  and  explain 

the  exam  and,  in  some  cases,  uncover  errors  in  the  written 

protocol  without  the  risk  of  biasing  the  responses. 

In  summary,  the  administration  process  was  designed  to 

systematically  obtain  data  using  the  following  protocol 

methodology : 

1.   Verbal  protocol — to  clarify  infrequent  ambiguities 
in  the  written  summaries. 
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2.  Graphic  protocol — to  illustrate  a  path  of  the 
subject's  thought  processes. 

3.  Written  suminaries — to  obtain  a  formalized,  step-by 
step  analogy  of  the  problem. 

4.  Interaction  with  experimenter — to  obtain  additional 
clarification  when  imagery  explanations  were 
ambiguous  and  when  steps  were  omitted  during  the 
written  summaries. 

The  importance  of  this  data  collection  method  is  demon- 
strated in  the  data  analysis  phase  of  the  procedural  model. 
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III.   DEVELOPMENT  OF  A  RESPONSE  EVALUATION  TECHNIQUE 

A  response  is  a  unit  of  behavior  and  the  building  block 
of  complex  performances.   The  flick  of  an  eye  and  the  twitch 
of  a  finger  are  examples  of  simple  responses;  eating,  walk- 
ing, speaking  and  reading  are  all  instances  of  more  complex 
responses.  A  primary  objective  of  educational  technology 
is  the  guidance  of  individual's  responses.   To  accomplish 
this  objective,  the  educator  must  first  define  and  enumer- 
ate the  components  of  the  perfonnance,  that  is,  the 
responses,  that  he  wishes  to  produce.   It  is  then  possible 
to  arrange  the  stimulus  conditions  which  will  result  in  the 
desired  response.   It  also  becomes  possible  to  develop 
objective  measures  of  the  frequency  and  accuracy  of  the 
response. 

The  consequences  of  a  student's  response  are  extremely 
important  in  learning.   The  events  which  follow  the  occur- 
rence of  a  response  have  an  effect  upon  future  behavior. 
Examples  of  such  response  consequences  are  reward,  punish- 
ment, and  knowledge  of  results.   The  reward  or  other  stimu- 
lation provided  to  the  learner  immediately  after  responding 
is  a  significant  factor  which  seems  to  determine  whether 
learning  takes  place.   The  occurrence  of  certain  consequen- 
ces of  behavior  that  are  effective  in  producing  and 
maintaining  behavior  is  called  reinforcement  by  behavioral 
psychologists.   In  an  educational  situation,  the  instructor 
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controls  the  consequences  of  the  student's  behavior.   Since 
these  consequences  determine  whether  the  student  learns, 
the  educator  will  want  to  maximize  those  consequences  which 
facilitate  learning. 

Definition  of  an  effective  and  efficient  procedural 
model  for  a  response  evaluation  methodology  began  with  an 
analysis  of  the  characteristics  of  the  instructional  environ- 
ment.  This  examined  specifically  the  environment  defined 
by  the  evaluation  of  course  mastery,   and  the  types  of  in- 
structional problems  for  which  a  response  evaluation  process 
could  provide  solutions  in  this  environment. 

The  procedural  model,  as  derived,  consists  of  seven 
major  components,  each  with  a  number  of  constituent  steps. 
The  components  are  selection  of  target  content,  design  of 
problem  solving  task,  selecting  protocol  methodology,  pre- 
testing, refining  the  task,  administering  the  final  version 
of  the  task,  and  analysis  of  results.   The  full  model  is 
shown  in  figure  1.   The  model's  seven  component  steps  are 
described  in  the  following  paragraphs. 

A.   SELECTION  OF  TARGET  CONTENT 

The  first  component  of  the  procedural  model  (see  figure 
2)  assumes  the  presence  of  a  main  track  of  non-CAI  materials 
and  defines  criteria  for  selecting  those  materials  for  which 
alternative  CAI  modules  are  to  be  developed.   The  steps  in 
this  component  do  not  constitute  a  precise  algorithm  for 
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SELECTION  OF  TARGET  CONTENT 

1.  Calculus 

2,  Upper  bound  of  Sets 


DESIGN  TEST 

1.  Preliminary  Knowledge  of  System  and  Problem  Solving  Task 

2.  Determine  All  possible  responses  that  are  to  be  achieved 

a.  correct 

b.  incorrect 


SELECT  PROTOCOL  METHODOLOGY 

1.  Subjects  to  draw  visual  Mental  Imagery 

2.  Subjects  to  use  verbal  responses 


PRETEST:   12  Subjects 


] 


MODIFY  TEST  VERSION 

1.  Delete  Problems  with  more  than  one  theoretical  response 

2.  Delete  Problems  with  a  multitude  of  incorrect  responses 


ADMINISTER  FINAL  VERSION  OF  THE  TEST 

1.  Sixty-one  subjects 

2.  Balance  subjects  by  ability,  field,  sex 


ANALYZE  RESULTS 


FIGURE  1. 


This  flowchart  summarizes  the  main  steps  in  the 
experimental  design  and  administration  of  a 
response  evaluation  procedure. 
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content  selection.   Rather  the  intent  is  to  identify  a 
niimber  of  factors  which  should  be  considered  in  determining 
target  content  selection.   Few  hard  and  fast  rules  can  be 
defined,  and  value  judgements  play  a  major  role  in  the 
decision  process. 

The  first  step  in  the  selection  of  target  content  is  to 
select  a  course  or  courses  where  the  use  of  CAI  will  be 
beneficial  to  the  training  process.   This  should  be  a 
school-level  decision  based  on  considerations  as  training 
cost  per  student,  instructor  student  ratios,  and  types  of 
training  materials  used. 

The  principal  consideration  must  be  to  choose  a  course 
where  CAI  materials  can  improve  motivation,  and  where  the 
unique  capabilities  of  CAI  will  provide  needed  information. 

After  selecting  a  course,  the  next  step  is  to  select  a 
specific  concept.   Here,  the  selection  considerations  are 
the  importance  of  the  particular  concept,  concept  failure 
rates,  variability  of  concept  mastery  test  scores,  and  aver- 
age times  and  variability  of  times  to  complete  the  concept. 

When  the  two  selection  steps  (course  and  concept)  are 
completed,  it  must  be  determined  if  CAI  is  feasible  for  the 
tentative  selections.   This  is  where  preliminary  knowledge 
of  the  system  plays  an  important  role.   Possible  reasons  for 
determining  non- feasibility  of  CAI  include  necessity  for 
graphics  or  simulations  which  are  beyond  the  capabilities  of 
the  available  system  and  personnel,  or  too  few  concept 

28 


selections  to  justify  hardware  or  development  cost. 

With  the  final  determination  of  CAI  feasibility,  the 
educator  is  now  able  to  define  and  enumerate  the  components 
of  performance,  in  other  words,  the  response  he  wishes  to 
produce  for  a  given  task.   The  educator  is  ready  to  begin 
the  next  component  of  the  procedural  model,  the  design  of 
the  problem-solving  task. 

B.   DESIGN  OF  PROBLEM  SOLVING  TASK 

The  steps  of  the  design  and  the  development  of  a 
problem-solving  task  component  is  shown  in  figure  3.   It 
assumes  that  the  course  and  concept  selection  together  with 
their  CAI  feasibility  have  already  been  determined. 

In  starting  to  develop  a  new  teaching  session,  an  edu- 
cator must  be  given  a  certain  amount  of  basic  preliminary 
information  about  the  material  and  system  to  be  used.   This 
may  be  provided  in  the  form  of  text,  drawings,  diagrams, 
moving  pictures,  or  any  combination  of  these  modes.   This 
process  will  ensure  that  the  finished  materials  avoid  such 
problems  as  omitting  necessary  content,  covering  objectives 
incompletely  or  at  an  incorrect  level,  incorrect  or  inade- 
quate emphasis  on  particular  subjects  or  concepts,  incom- 
plete material  supplements,  poorly  designed  teaching 
strategies,  and  tests  which  do  not  address  the  objectives. 
Once  the  workings  of  the  system  and  the  basics  of  the 
materials  are  understood,  development  of  the  task  to  obtain 
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proper  responses  from  the  user  and  the  system  may  begin. 

After  this  initial  phase,  the  design  of  the  problem- 
solving  task  should  begin  with  a  careful  analysis  of  the 
instructional  requirements.   The  forming  of  the  instrument 
to  evaluate  a  particular  concept  is  obtained  through  the 
educational  process  of  an  instructors  preliminary  knowledge 
of  the  subject  matter.   In  this  phase,  it  is  important  that 
the  test  has  face  validity  for  the  objectives  which  addres- 
ses the  particular  concept.   It  is  also  important  that  the 
problem-solving  task  cover  the  specific  concept  matter  at 
the  appropriate  difficulty  levels.   Since  this  instrument  is 
the  principal  element  which  will  guide  the  response  evalua- 
tion process,  it  must  fully  represent  the  intent  of  the 
educator  to  assure  adequate  performance.   Any  revisions  must 
be  reviewed  by  the  educator  to  insure  the  technical  accuracy 
of  the  content  changes . 

Unless  these  first  two  steps  are  successfully  completed, 
all  subsequent  efforts  may  be  jeopardized.   For  example,  if 
the  problem-solving  task  is  so  designed  that  it  will  not 
evaluate  the  particular  concept,  the  work  in  the  evaluation 
of  principal  incorrect  responses  for  that  particular  concept 
will  be  largely  wasted. 

The  final  step  of  the  design  of  the  problem-solving  task 
includes  the  determination  of  the  proper  responses  desired 
and  the  possible  major  incorrect  responses  that  may  be 
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obtained.   Since,  the  number  of  possible  incorrect  responses 
for  any  particular  concept  is  theoretically  infinite,  this 
process  will  only  serve  as  an  initial  starting  point  in  the 
identification  of  the  principal  incorrect  forms  of  that  con- 
cept that  is  taught. 

C.   SELECTION  OF  PROTOCOL  METHODOLOGY 

The  protocol  methodology  for  the  administering  and  eval- 
uation of  the  problem-solving  task  (see  figure  4)  are  given 
as  a  set  of  rules.   Revision  of  the  protocol  may  be  made  at 
any  time  it  is  determined  that  it  is  non- feasible  for  a 
particular  application.   The  rules  are  as  follows: 

1.  Selection  of  students  for  use  on  the  problem-solving  task 
should  be  on  a  volunteer  basis.   Using  volunteers  rather  than 
individuals  ordered  to  participate  in  the  problem-solving 
task  will  avoid  some  possible  attitude  problems. 

2.  Volunteers  should  be  selected  on  their  qualification  for 
the  particular  concept  being  tested.   They  should  be  se- 
lected to  provide  a  reasonably  balanced  representation  by 
sex,  specialization  and  professional  level. 

3.  The  problem-solving  task  should  be  conducted  outside  the 
classroom  and  should  be  informal.  Attempting  to  work  within 
the  classroom  is  likely  to  disrupt  regular  classroom  activi- 
ties.  In  addition,  it  has  been  found  that  the  problem- 
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solving  task  should  be  administered  to  one  test  subject  at 
a  time.   Group  testing  tended  to  inhibit  the  subject's  will- 
ingness to  verbally  report  their  thought  processes. 

4 .  The  examiner  should  be  available  to  answer  questions , 
provide  guidance  where  necessary,  and  note  areas  where 
students  have  difficulty. 

5.  Finally,  and  most  important,  the  test  subjects  are  to 
learn  the  particular  concept  and  apply  it  to  a  number  of 
practice  problems.   During  the  problem-solving  task,  the 
test  subjects  are  to  verbally  report  their  thought  proc- 
esses and  draw  and  describe  mental  images,  if  any  occurred. 
In  this  way,  the  subject's  thought  processes  for  a  partic- 
ular concept  may  be  determined  by  analyzing  the  result  and 
the  sequence  of  steps  that  were  used  to  obtain  that  result. 

The  above  rules  provide  effective  guidelines  for  an 
efficient  and  standard  administration  and  evaluation  of 
the  problem-solving  task.   It  is  noted  that  verbal  re- 
ports of  a  subject's  thought  processes  may  be  tape  re- 
corded for  accuracy  and  historical  references.   This  is 
only  required,  however,  if  it  has  been  found  that  test 
subjects  tend  to  lean  towards  verbal  reporting  vice  the 
written  method.   Continual  evaluation  of  the  methodology 
must  be  made  during  the  problem-solving  task  for  any 
particular  situation. 
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D.   PRETEST 

The  fourth  component  of  the  procedural  model  (see 
figure  5)  assumes  that  a  dedicated  amount  of  time  has  al- 
ready been  spent  in  the  selection  and  development  of  a 
problem-solving  test.   The  purpose  of  this  phase  is  not  so 
much  directed  at  evaluating  the  response  effectiveness  of 
the  problem-solving  task  as  it  is  at  locating  errors  of 
content  and  logic  which  were  overlooked.   Typically,  the 
number  of  students  tested  would  not  be  sufficient  to  answer 
questions  concerning  the  incorrect  concepts  that  the  student 
learns,  but  do  provide  an  avenue  by  which  the  educator  can 
learn  if  the  materials  are  easily  understood  by  their  in- 
tended audience.   The  pretest  provide  answers  to  questions 
such  as  the  following.   Are  there  inconsistencies  between 
the  concept  and  the  problem-solving  task?  Are  directions  to 
the  student  clear  enough  to  be  followed?   Is  wording  clear? 
Is  the  problem-solving  task  too  long  to  hold  the  student's 
attention? 

The  pretest  should  generally  be  administered  to  12-15 
students,  but  the  number  will  vary  with  the  number  of  errors 
found  early  in  the  pretest.   If  many  errors  are  found  with, 
for  example,  the  first  four  or  five  students,  it  would  be 
preferrable  to  suspend  the  pretest  until  the  problems  have 
been  resolved.   If  available  manpower  permits,  it  may  be  de- 
sirable to  supplement  review  of  the  pretest  by  other  instruc- 
tional personnel  for  the  purpose  of  locating  grammatical  and 
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typographical  errors  and  problems  with  clarity. 

The  next  step  pertains  to  data  analysis.   With  the  use 
of  imagery/  as  specified  in  step  3  of  the  procedural  model, 
principal  incorrect  forms  of  the  concept  that  a  student 
learns  may  now  be  initially  determined.   These  incorrect 
concept  forms  are  to  be  compared  to  the  theoretical  incorrect 
forms  established  in  step  2  of  the  procedural  model.   After 
comparison  of  both  the  theoretical  and  diagnosed  forms  of  the 
principal  incorrect  responses ,  the  educator  is  now  ready  to 
begin  the  next  component  of  the  procedural  model,  modifica- 
tion of  the  problem  solving  task. 

E.   MODIFICATION  OF  THE  PROBLEM  SOLVING  TASK 

This  component  (see  figure  6)  of  the  procedural  model  is 
designed  to  modify  or  correct  any  errors  and  ambiguities  re- 
vealed by  the  pretest.   It*s  purpose  is  to  evaluate  the 
instructional  effectiveness  of  the  materials  to  determine 
the  specific  areas  of  weakness.   Problems  which  have  more 
than  one  theoretical  incorrect  response  or  too  many  diagnosed 
responses,  such  that  a  specific  incorrect  concept  cannot  be 
applied  to  the  problem,  are  to  be  evaluated  and  deleted  from 
the  problem-solving  task.   If  it  is  found  that  the  problem- 
solving  task  has  too  many  unresolvable  problems,  it  must  be 
determined  if  the  particular  subject  matter  is  actually 
testable.   If  not,  the  procedural  process  for  a  response 
evaluation  procedure  terminates  at  this  point,  otherwise  the 

educator  is  ready  to  begin  the  next  component  of  the 
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procedural  model,  administration  of  the  problem-solving  task 
for  formative  evaluation. 

F.  ADMINISTERING  OF  THE  PROBLEM  SOLVING  TASK 

The  sixth  component  of  the  procedural  model  (see  figure 
7)  assumes  that  the  problem-solving  task  has  now  been  per- 
fected and  is  ready  to  be  used  to  evaluate  the  incorrect 
responses  for  the  particular  concept.   A  reasonably  balanced 
population  of  test  subjects  by  sex,  specialization  and  pro- 
fessional levels  are  to  be  provided.   The  test  is  to  be 
administered  in  the  same  methodology  as  was  used  in  the  pre- 
testing of  the  instrument. 

The  final  study  should  be  administered  to  20-30  students, 
but  the  number  will  vary  depending  on  the  particular  con- 
cept that  is  being  evaluated.   Successful  completion  of  this 
component  moves  the  sequence  of  the  response  evaluation 
process  to  the  last  component  of  the  procedural  model,  data 
analysis. 

G.  DATA  ANALYSIS 

The  analysis  of  the  results  appears  in  the  final  chap- 
ters.  However,  a  brief  outline  may  be  useful  in  interpret- 
ing the  data  analysis  (see  figure  8) . 

The  first  step  examines  the  incorrect  form  of  the 
responses  by  subject  and  by  problem.   The  influence  of  abil- 
ity level,  sex  and  type  of  problem  are  considered  next.  Final- 
ly, the  data  will  be  analyzed  as  its  applicability  to  CAI . 
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FIGURE  7.  Administering  Problem-Solving  Task 
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FIGURE  8.   Data  Analysis 
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IV.   DATA  ANALYSIS 

The  final  study  included  sixty-one  subjects.   Thirty- 
seven  of  these  were  obtained  from  prior  research 
(Weissinger-Baylon,  1978)  and  twenty-four  (see  Appendix  A) 
were  obtained  during  the  course  of  this  study.   They  provide 
a  group  reasonably  balanced  by  sex,  specialization,  profes- 
sional level,and  most  important,  ability  to  understand  the 
particular  concept — the  upper  bound  of  a  set. 

The  test  group  was  comprised  of  23%  females  and  77% 
males.   Of  these,  33%  were  graduate  or  undergraduate  stu- 
dents in  the  mathematical  sciences  and  67%  were  under- 
graduates or  non-mathematical  science  graduates.   All 
subjects  were  volunteers,  and  the  time  required  to  do  this 
experiment  varied  from  10-29  minutes  per  subject.   The  final 
test  instrument  consisted  of  a  definition,  a  practice  prob- 
lem, and  nine  problem-solving  sets  (see  figure  9) . 

As  is  true  in  the  analysis  and  evaluation  of  any  problem- 
solving  task,  the  principal  challenge  lies  in  summarizing 
the  protocol  data.   Using  data  from  671  problems  obtained 
during  this  research,  the  response  evaluation  technique  for 
the  upper  bound  of  a  set  concept  was  evaluated  from  three 
different  standpoints:   (a)  evaluation  of  its  ability  to 
isolate  the  major  incorrect  responses  for  a  particular 
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FINAL  TASK  DESCRIPTION 


Set  Nimiber 

Set 

Definition 

Let  E  be  a  subset  of  the  real  niraibers,  R. 
An  element,  m,  of  R  is  said  to  be  an  upper 
bound  of  E  if  X  is  equal  to  or  less  than  m 
for  every  x  in  E 

Practice 
Problem 

Let  E  be  the  set  [1,2,3,4],  i.e.  the  set 
consisting  of  the  first  four  positive 
integers.  Does  E  have  an  upper  bound? 

Set  2 

The  set  of  natural  numbers,  N  (i.e.,  the 
positive  integers) . 

Set  4 

A  finite  set. 

Set  5 

(l,  1/2,  1/3,  ...} 

Set  6 

The  set  of  real  numbers,  R. 

Set  11 

The  non-negative  numbers 

Set  13 

The  closed  interval  [3,5J 

Set  17 

{/2,/3,/3',/B,/^,;7,  ...} 

Set  18 

{tan  X,  where  x  =Tf  ,  2rf,  3rr,   4rr,   S/f,   Sir,    ...} 

Set  21 

{1,2,4,8,16,  ...} 

Figure  9.   The  sequence  of  steps  to  perform  the  task  consisted  of: 
(1)  reading  the  definition,  (2)  and  then  solving  the  practice  problem. 
Each  subject  was  then  instructed  to  determine  whether  or  not  each  of 
the  sets  had  an  upper  bound.   During  the  process  of  the  problem  solving 
task,  subjects  were  asked  to  provide  verbal  and  written  reports  of 
their  thought  processes,  and  draw  mental  pictures,  if  any  occurred. 
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concept,   (b)  evaluation  of  response  error  statistics  by 
type  of  set,  and  type  of  set  between  ability  level  and  sex 
and, (c)  evaluation  of  subjects  response  pattern.   Results 
of  each  of  these  three  evaluation  efforts  are  reported  in 
the  following  subsections. 

A.   ISOLATION  OF  MAJOR  INCORRECT  RESPONSES 

Prior  research  has  dictated  the  existence  of  an  in- 
finite number  of  theoretical  correct  and  incorrect  re- 
sponses for  any  particular  concept,  depending  on  the 
nature  of  the  behavior  and  the  level  of  skill  of  the  indi- 
vidual.  JTaber,  Glaser,  Schaeffer,  1965,  p.  19]  .   Keeping 
this  fact  in  mind,  the  isolation  of  major  incorrect 
responses  becomes  of  the  utmost  importance  during  the 
final  evaluation  of  any  problem-solving  task. 

From  the  educators  preliminary  knowledge  of  the  con- 
cept material  and  the  initial  analysis  of  the  pretest 
results — procedural  model,  step  two  and  four  respectively, 
seven  major  incorrect  responses  or  incorrect  concept  forms 
were  identified  as  being  possible  for  the  upper  bound 
concept.   They  were  classified  as  follows: 

1.  Positive  infinity  is  the  upper  bound  for  all 
sets . 

2.  Infinite  number  of  points  has  no  upper  bound. 

3.  An  infinite  set  has  no  upper  bound. 

4.  The  upper  bound  has  to  be  a  member  of  the  set, 
if  the  set  is  to  have  an  upper  bound. 
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5.  Misreading  of  the  problem. 

6.  Misunderstanding  the  problem. 

7.  Other  (Ignoring  the  problem,  unable  to  do 
problem,  etc.) 

Analysis  of  the  above  incorrect  response  concepts 
indicated  that  incorrect  concept  forms  two  and  three 
(infinite  number  of  points  has  no  upper  bound  and  infinite 
set  has  no  upper  bound,  respectively)  were  very  similar  in 
construct  and  may  be  combined  to  state  that:   "no  upper 
bound  exists  for  an  infinite  set."   In  addition,  in- 
correct concept  form  six  (misunderstanding  the  problem) 
may  be  placed  in  the  incorrect  concept  form  seven  (other) 
category,  since  the  relationship  between  the  two  concept 
forms  were  similar  in  nature.   This  initial  detailed 
analysis  of  all  the  possible  major  concept  forms  resulted 
in  the  isolation  of  five  incorrect  concept  forms  for  the 
particular  upper  bound  concept. 

Application  of  the  five  incorrect  concept  forms  to  the 
problem-solving  sets,  illustrated,  theoretically,  the 
errors  that  an  individual  would  or  should  obtain  with  each 
particular  incorrect  concept  form  that  he  had  learned  (see 
figure  10) .   It  must  be  noted  that  the  final  test  instru- 
ment was  designed  and  modified  (procedural  model,  step 
five)  so  that  no  more  than  one  theoretical  incorrect 
concept  applied  to  each  problem-solving  set. 
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Using  the  above  information,  the  final  test  results  were 
analyzed  and  the  errors  were  evaluated  in  regards  to  the 
five  major  incorrect  concept  forms  (see  figure  11) .   The 
analysis  indicated  that  94  percent  of  the  errors  fell 
into  the  particular  error  category  that  was  determined, 
from  the  theoretical  application  of  incorrect  concept 
forms ,  to  produce  an  error  for  the  individual  problem- 
solving  set.   In  addition,  80  percent  of  the  errors  fell 
into  just  two  major  incorrect  concept  categories:   plus 
infinity  is  the  upper  bound  of  everything  and  an  in- 
finite set  has  no  upper  bound. 

It  is  reassuring  to  note  that  the  absence  of  response 
errors  for  the  major  incorrect  concept  form  "the  upper 
bound  must  be  an  element  of  the  set"  indicated  that  the 
total  number  of  incorrect  response  forms  for  this  par- 
ticular concept  may  be  further  reduced  to  only  four 
major  categories.   This  is  a  reduction  of  43  percent  from 
the  number  initially  determined  in  the  early  stages  of  the 
response  evaluation  model. 

B.   SUBJECT  AND  SET  TYPE  EVALUATION 

Results  of  the  "subject  to  set  evaluation"  is  discussed 
in  two  parts:   evaluation  of  type  of  sets,  and  evaluation 
of  subject,  ability  level  and  sex  between  type  of  set. 

The  errors  for  the  particular  sets  were  tabulated  in 
figure  12.   The  sets  are  classified  as  being  either 
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"Abstract,"  those  that  may  require  interpretation,  for 
example:   "The  non-negative  numbers,"  and  "Concrete,"  those 
which  sxibjects  can  analyze  directly,  for  example: 
"{1,2,4,8,16,,..}."   Statistical  analysis  indicated  that  the 
error  rates  for  the  abstract  sets  were  considerably  higher 
than  the  concrete  sets  (72%  and  28%,  respectively).   The 
difference  may  be  attributed  to  three  significant  factors : 
the  final  instrument  consisted  of  five  abstract  sets  com- 
pared to  only  four  concrete  sets;  the  error  rate  for  set 
eighteen  is  abnormally  high;  and  due  to  the  inherent 
nature  of  the  abstract  type  of  set. 

Deletion  of  set  eighteen  from  the  test  instrimient  to 
remove  the  problem  solving  set  with  the  abnormally  high 
error  rate  and  to  provide  an  equal  distribution  of  abstract 
and  concrete  sets,  still  resulted  in  an  error  rate  differ- 
ence between  the  two  types  of  sets  of  60  percent  for 
abstract  sets  and  40  percent  for  concrete  sets.   Therefore, 
is  the  inherent  nature  of  the  type  of  sets  a  significant 
factor  on  the  type  of  response?   Consider  for  example  set 
eleven: 

Set  11.   The  non-negative  numbers. 
Ten  subjects  answered  it  incorrectly:   six  by  incorrectly 
applying  the  upper  bound  definition,  and  four  by  misunder- 
standing the  term  "non-negative."   Now,  consider  a  mathe- 
matically similar  task,  set  twenty-one: 
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Set  21.   {1,2,4,8,16,..  .}. 
Both  sets  are  infinite  with  the  only  difference  that  set 
eleven  is  written  in  abstract  terms  while  set  twenty-one 
is  written  in  concrete  terms  consisting  of  five  elements. 
Since  the  differences  between  the  sets  are  minor,  error 
rate  comparison  is  surprising.   Indicating  that  the  in- 
herent nature  of  a  problem-solving  set  is  a  significant 
factor  in  the  type  of  response  that  may  be  obtained. 

Next,  error  response  for  subject,  ability  level,  sex 
and  type  of  set  (see  figure  13)  is  analyzed  and  provided 
below: 

1.  It  was  found  that  the  overall  response  error  rate  for 
the  problem-solving  instrument  was  11.7  percent.   Even 
though  the  test  was  designed  to  be  easy  and  administered 
to  subjects  who  had  the  ability  to  understand  the  particu- 
lar concept,  this  indicates  that  individuals,  as  problem 
solvers,  are  often  unreliable  for  objectively  easy  tasks. 

2.  There  was  a  significant  difference  of  3.3  percent  re- 
sponse error  rate  between  the  "mathematical  science"  group 
and  the  "other"   group.   The  "mathematical  science"  group 
consisted  of  graduates  from  the  Naval  Postgraduate  School 
and  Stanford  University  and  undergraduates  from  Colorado 
State  University  in  the  mathematical  sciences  (Mathematics, 
Computer  Science,  Statistics,  and  Operations  Research). 
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SUBJECT  IDENTIFIERS,  SEX,  FIELD,  LEVEL,  AND  ERRORS 


1  Sub: 

lect 

Sex 

Field 

MATH 

YEAR 

Errors 

1  Niainber 

SCIENGE 

No. 

% 

SI 

CD. 

M 

GSM 

GRAD 

1 

11 

S2 

L.M. 

F 

GS 

X 

GRAD 

2 

22 

S3 

T.V. 

M 

GSM 

GRAD 

1 

11 

S4 

R.W. 

M 

GSM 

GRAD 

2 

22 

S5 

S.L. 

M 

GSM 

GRAD 

0 

0 

S6 

P.H. 

M 

GS 

GRAD 

3 

33 

S7 

Y.K. 

M 

GSM 

X 

GRAD 

4 

44 

S8 

R.F. 

M 

D.P. 

UND 

1 

11 

S9 

F.K. 

M 

D.P. 

UND 

4 

44 

SIO 

D.M. 

F 

MATH 

X 

UND 

1 

11 

Sll 

S.W. 

M 

M.E. 

UND 

1 

11 

S12 

B.N. 

M 

G.E. 

UND 

3 

33 

S13 

E.G. 

F 

MATH 

X 

UND 

0 

0 

S14 

G.J. 

M 

C3 

GRAD 

0 

0 

S15 

R.G. 

M 

D.P. 

UND 

1 

11 

S16 

K.G. 

M 

D.P 

UND 

1 

11 

S17 

E.T. 

M 

D.P. 

X 

UND 

2 

22 

S18 

J.L. 

M 

G.S. 

GRAD 

0 

0 

S19 

L.N. 

F 

G.E. 

UND 

1 

11 

S20 

T.M. 

H 

D.P. 

X 

UND 

4 

44 

S21 

A.G. 

M 

D.P. 

GRAD 

1 

11 

S22 

J.T. 

M 

G.S. 

UND 

2 

22 

S23 

J.B. 

M 

M.E. 

UND 

0 

0 

S24 

S.M. 

M 

M.E. 

UND 

2 

22 

T15 

R.E. 

M 

O.R. 

X 

GRAD 

0 

0 

T16 

J.M. 

M 

G.S. 

X 

GRAD 

5 

55 

T17 

P.M. 

M 

M.D. 

M.D. 

3 

33 

T18 

L.W. 

M. 

MATH 

X 

Ph.D. 

0 

0 

T19 

N.B. 

M 

MATH 

X 

GRAD 

1 

1 

T20 

R.P. 

M 

C.S. 

X 

GRAD 

0 

0 

T21 

R.R. 

M 

MATH 

X 

GRAD 

0 

0 

T22 

J.F. 

M 

M.D. 

X 

GRAD 

0 

0 

T23 

D.D. 

M 

STAT 

X 

GRAD 

0 

0 

T24 

A.K. 

M 

MATH 

X 

GRAD 

1 

11 

Figure  13.   The  error  values  are  for  nine  sets. 
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SUBJECT  IDENTIFIERS,  SEX,  FIELD,  LEVEL,  AND  ERRORS 


Sxibject 

Sex 

Field 

Math 

Year 

Errors 

Number 

Science 

No. 

% 

T25 

H.S. 

M 

MATH 

X 

GRAD 

0 

0 

T26 

S.K. 

M 

MATH 

X 

GRAD 

0 

0 

T27 

B.B. 

M 

BIOL 

UND 

0 

0 

T28 

J. A. 

F 

Phys/CS 

GRAD 

2 

22 

T29 

S.L. 

F 

Lang 

GRAD 

1 

1 

T30 

P.W. 

M 

MATH 

X 

Ph.D. 

2 

22 

T31 

J.H. 

M 

MATH 

X 

GRAD 

0 

0 

T32 

A.S. 

M 

STAT 

X 

GRAD 

0 

0 

T33 

G.K. 

M 

STAT 

X 

GRAD 

1 

11 

T34 

R.G. 

M 

PSY 

UND 

0 

0 

T35 

S.N. 

M 

MATH 

X 

GRAD. 

0 

0 

T36 

J.N. 

M 

C.E. 

GRAD 

3 

33 

T37 

J.K. 

M 

ENG 

AB 

1 

11 

T38 

R.B. 

M 

M.E. 

UND 

0 

0 

T39 

W.P. 

M 

M.E. 

UND 

0 

0 

T40 

D.A. 

M 

PSY 

UND 

1 

1 

T41 

N.A. 

F 

EDUC 

GRAD 

- 

- 

T42 

J.F. 

F 

C.E. 

UND 

0 

0 

T43 

B.B. 

M 

UNDECL 

GRAD 

0 

0 

T44 

B.E. 

M 

PHYS 

UND 

4 

44 

T45 

J.R. 

F 

UNDECL 

UND 

0 

0 

T46 

N.Z. 

F 

UNDECL 

UND 

1 

11 

T47 

V.L. 

F 

UNDECL 

UND 

0 

0 

T48 

L.H. 

F 

UNDECL 

UND 

1 

11 

T49 

C.H. 

F 

MATH 

UND 

0 

0 

T50 

J.H. 

M 

C.E. 

UND 

0 

0 

T51 

K.T. 

F 

PHYS 

GRAD 

0 

0 

T52 

J.G. 

F 

UNDECL 

UND 

0 

0 

Figure  13  (con't)   The  error  values  are  for  nine  sets. 
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The  "other"  group  consisted  of  Naval  Postgraduate  School 
and  Stanford  University  graduates  and  Colorado  State 
University  undergraduates  in  curriculums  other  than  the 
mathematical  sciences.   The  response  error  rates  for  the 
groups  were:   9.4  percent  for  the  "mathematical  sciences," 
and  12.7  percent  for  the  "other"  group.   This  significant 
response  error  difference  provided  a  strong  indication  that 
additional  years  of  specialized  training  does  provide  an 
improved  response  performance  for  technically  inclined 
concepts . 

3.  The  response  error  rate  for  males  was  12.5  percent, 
for  females  8.7  percent.   The  lower  response  error  rate 
for  females  is  very  surprising,  since  today's  society  is 
still  geared  towards  the  higher  technical  training  of  the 
male  counterpart.   The  error  rate  difference,  however,  may 
be  explained  by  the  unequal  test  population  of  females 
compared  to  males. 

4.  As  indicated  in  the  set  type  evaluation,  the  error  re- 
sponse rate  for  the  "Abstract"  set  category  was  found  to 
be  considerably  higher  than  the  "Concrete"  set  category — 
15.1  percent  and  7.4  percent,  respectively.   This  is  a 
result  of  the  inherent  nature  of  the  problem.   It  has  been 
found  that  individuals  relate  better  to  problems  which  may 
immediately  be  visualized.   Thereby,  reducing  the  amount 
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of  time  required  for  the  thought  process  and  thus,  minimize 
the  chance  for  error. 

5.   Response  error  rates  for  subject's  ability  level  between 
set  type  and  subject's  sex  between  set  type  is  given  in 
figure  14  and  figure  15  respectively.   The  figures  indicate 
that  mathematical  science  students  are  better  prepared  for 
technical  concepts  provided  in  concrete  terms,  however, 
fail  to  provide  a  better  score  for  abstract  type  problems. 
Moreover,  the  female  is  apt  to  provide  better  responses  than 
their  male  counterpart  for  the  specific  set  type  categories. 

The  above  data  analysis  for  the  different  categories 
indicates  that  a  wide  disparity  exists  between  and  within 
the  groups  that  were  looked  at.   A  split  halves  reliability 
coefficient,  to  determine  if  the  nine  sets  constitute  an 
acceptable  instrument,  was  obtained  by  regressing  total 
errors  of  even  numbered  subjects  on  the  total  for  odd 
numbered.   The  coefficient  was  r  =  .66,  suggesting  that 
the  problem-solving  task  was  an  acceptable  instrument. 

C.   SUBJECTS  RESPONSE  PATTERN 

Analysis  of  figures  ten  and  eleven,  indicated  that  a 
definite  mastery  of  an  incorrect  concept  existed  for  some 
test  subjects,  while  for  others  it  was  not  so  easy  to 
determine  an  error  pattern  (see  figure  16) . 
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Math  Sci. 
n  =  20 


Other 
n=41 


Concrete 
sets 

Abstract 
sets 


,050 


130 


.085 


.161 


Figure  14  Response  error  rates  between  ability  level  and 
set  type 


Male 
n  =  47 


Female 
n  =  14 


Concrete 
sets 

Abstract 
sets 


.074 


.166 


.071 


.100 


Figure  15  Response  error  rates  between  sex  and  set  type 
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SUBJECT  RESPONSE  ERROR  PATTERN 

s 
u 

INCORRECT  CONCEPT  FORM 

Plus  infinity  is  the 

An  infinite  set  has 

no 

The  upper  boiind  must 

B 

upper  bound  for  all 

upper  bound 

be  an  element  of  the 

J 

sets 

set 

E 

C 

!Jo.          No. 

No.       No. 

No .       No . 

T 

rheor .      Sub j . 

% 

Theor .     Sub j . 

% 

Theor .   Svib  j  .     % 

Errors     Errors 

Errors     Errors 

E  rrors   Errors 

S2 

3         1 

33 

S3 

3          1 

33 

S6 

5          3 

60 

S7 

5          3 

60 

3          1 

33 

S8 

3          1 

33 

S9 

5          4 

80 

S12 

5          3 

60 

S17 

3          1 

33 

S20 

3          3 

100 

S21 

3          1 

33 

S22 

3          2 

66 

S24 

3          2 

66 

T16 

5          4 

80 

T17 

3          2 

66 

T28 

3          2 

66 

T30 

5          2 

40 

T33 

3          1 

33 

T36 

5         3 

60 

T37 

3          1 

33 

T40 

3          1 

33 

T44 

5         4 

80 

T48 

3          1 

33 

Total 

40         26 

45         21 

0        0 

Avg 

3.25      65% 

1.35 

47% 

0      0% 

Figure  16 .  The  number  of  siabject  errors  are  compared  to  the  number 
of  problem  sets  that  should  have  theoretically  produced   error  with 
the  particular  incorrect  concept  form. 
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Consider,  for  example,  the  major  incorrect  concept 
form  "plus  infinity  is  the  upper  bound  for  all  sets." 
Subjects  S9,  T16  and  T44  responded  incorrectly,  using  the 
particular  incorrect  concept,  to  four  out  of  five  problem- 
solving  sets  which  were  theoretically  designed  to  produce 
errors  if  that  particular  incorrect  concept  form  was 
applied.   Their  test  results  definitely  indicated  a  mas- 
tery of  the  incorrect  concept  form  as  illustrated  in  the 
following  S9 ,  T16  and  T44  excerpts  for  set  eleven: 


S9 :   "Since  plus  infinity  is  the  largest  possible 
number,  it  must  be  the  upper  bound." 

T16:  "Bounded  by  continuum  i.e.,  infinity  of  con- 
tinuum greater  than  infinity  of  natural 
numbers . " 

T44:  "Then  I  remember  very  strongly  that  infinity 
is  the  largest.   So  u.b.  is  infinity." 


The  other  missed  sets  for  the  particular  incorrect  re- 
sponse concept  produced  a  similar  type  of  incorrect  reason- 
ing process  from  the  three  test  subjects. 

Now  consider  subject  T30's  response  for  the  same 
incorrect  concept  form.   T30  responded  incorrectly  to  only 
two  out  of  five  problem  sets.   Excerpts  from  the  missed 
problem  sets  are  illustrated  below: 


Set  2:    "Nothing  can  be  to  right  of  all  of  them, 
therefore,  upper  bound  exists." 

Set  17:   "Square  root  is  unbounded  (i.e.,  off  to 
right).   Therefore,  upper  bound  exists." 
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The  response  was  implicit,  but  not  stated  as  clearly. 
Subject  T30  seemed  to  follow  the  correct  process  for  an 
infinite  set  being  unbounded,  but  then  stated  "therefore, 
upper  bound  exists." 

The  analysis  indicated  that  the  other  errors  produced 
by  the  test  subjects  followed  a  similar  pattern  as  was 
shown  above.   With  the  exception  of  S20,  no  other  test 
subject  answered  all  of  the  problem  sets,  designed  to  pro- 
duce an  error  for  a  particular  concept,  incorrectly. 
It  is  surprising  to  note  that  an  error  response 
pattern  for  each  subject  could  not  be  determined.   There- 
fore, is  it  meaningful  to  even  talk  of  error  patterns? 
Brunner,  Goodnow,  and  Austin  (1956)  discussed  the  role  of 
errors  and  indicated  that  a  notable  flexibility  of  the 
subject's  behavioral  pattern  exists.   Different 
strategies  will  be  formed  for  certain  information. 
Further  explanation  for  inconsistencies  in  error  response 
patterns  is  provided  in  the  following  statement: 

.  .  .  heuristics  do  not  guarantee  any  solution  at 
all;  all  that  can  be  said  for  a  useful  heuristic 
is  that  it  offers  solutions  which  are  good  enough 
most  of  the  time. 

[Peigenbaum  and  Feldman,  1963,  p.  6 J 
This  description  is  consistent  with  the  behavior  of 
the  subjects  error  patterns. 

The  error  pattern  for  test  subjects  with  the  in- 
correct concept  form  "the  upper  bound  must  be  an 
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element  of  the  set"  were  not  analyzed,  since  the  test 
instrument  was  designed  to  produce  no  errors  for  that 
concept — making  it  difficult  to  analyze  a  set  error 
pattern. 
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V.   CONCLUSIONS  AND  RECOMMENDATIONS 

The  proposed  complete  response  evaluation  approach 
described  here  appears  promising.   Test  results  have  dem- 
onstrated that  the  developed  technique  provides  the 
educator  with  a  method  to  structure  the  CAI  course  material 
towards  responses  which  a  student  is  more  likely  to  pro- 
vide and,  therefore,  will  prove  to  be  most  effective  for 
the  individual  student.  As  it  stands,  the  approach  is 
ready  for  use  by  CAI  lesson  writers  and  evaluation  per- 
sonnel.  This  final  section  summarizes  the  observed 
strengths  and  weaknesses  of  the  proposed  technique, 
suggests  specific  areas  for  further  development,  and  out- 
lines a  number  of  recommendations  for  utilizing  the 
response  evaluation  approach. 


A.   EFFECTIVENESS  OF  THE  RESPONSE  EVALUATION  PROCESS  AND 
SUGGESTIONS  FOR  FURTHER  DEVELOPMENT 


On  the  basis  of  test  results  discussed  in  the  preceding 
sections  of  this  report,  it  can  be  concluded  that  the  re- 
sponse evaluation  process  for  a  particular  concept  was 
reasonably  effective;  substantially  more  than  the  current 
processes  that  it  may  replace.   Use  of  this  approach  re- 
sulted in  the  dramatic  reduction  of  the  infinite  number  of 
possible  theoretical  incorrect  responses  for  a  particular 
concept  to  a  mere  three.   Also,  substantive,  although  not 
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striking,  improvements  in  the  methodology  for  administering 
the  final  instrument  were  discovered. 

It  must  be  admitted  that,  in  terms  of  one  of  the  more 
central  measures  of  success,  discovery  of  an  response  error 
pattern  for  each  individual  subject,  is  not  as  dramatic  as 
might  have  been  hoped  for.   A  set  error  pattern  was  dis- 
covered for  some  individuals,  but  could  not  be  generalized 
for  all  test  subjects. 

Two  major  processes  of  the  response  evaluation  approach, 
the  methodology  used,  and  the  error  response  pattern  will 
now  be  examined  and  discussed  in  some  detail. 

1.   Methodology  Used 

The  imagery  and  verbal  response  methodology  is  sound. 
Although  there  is  no  proof  that  imagery  dramatically  reduces 
problem-solving  errors  (Weissinger-Baylon,  1978) ,  it  is  an 
effective  method  for  evaluation  of  thought  processes  for  the 
particular  application  for  which  it  was  used.   It  must  be 
noted,  however,  that  some  problem-solving  tasks  make  heavy 
demands  on  imagery,  while  others  do  not.   This  is  not  the 
result  of  the  inability  of  the  subjects  to  produce  them, 
but  on  the  inherent  nature  of  the  task.   While  problems 
were  encountered  in  two  areas ,  they  were  more  of  the  nature 
of  management  problems  rather  than  problems  with  the  metho- 
dology per  se. 

First,  the  test  subjects  were  unaware  of  the  process  of 
using  imagery  in  their  problem-solving  task  and  were  re- 
luctant and  not  ready  to  accept  its  use.   Subjects  had  to 
be  continually  urged  to  put  down  any  mental  images,  if  and 
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when  they  occurred.   This  reluctance  to  describe  mental 
images  is  not  uncommon  in  the  imagery  controversy,  which 
dates  back  to  the  early  part  of  this  century.   The  imagery 
controversy  raged  between  psychologists  with  Holt  (1964) 
and  Titchener  (1926)  pleading  for  a  new  and  more  direct 
consideration  for  mental  imagery,  while  behaviorists  re- 
jected the  idea  as  unscientific.   It  was  not  until  the  mid- 
sixties  when  Shepard  (19  78)  and  others  discussed  the  im- 
portance of  imagery  in  the  problem-solving  role,  that  it 
was  more  readily  accepted  by  cognitive  psychologists. 
Therefore,  in  addition  to  the  recognized  newness  of  the  use 
of  imagery  in  problem-solving,  the  further  reluctance  of 
test  subjects  to  use  mental  images  to  solve  a  problem- 
solving  task  may  best  be  explained  by  Amheim: 

images  must  be  highly  abstract  since  the  mind  operates 
often  at  high  levels  of  abstraction.   But  to  get  at 
these  images  is  not  easy,   I  mentioned  that  a  good  deal 
of  imagery  may  occur  below  the  level  of  consciousness 
and  that  even  if  conscious,  such  imagery  may  not  be 
noticed  readily  by  persons  unaccustomed  to  the  awkward 
business  of  self-observation.   At  best,  mental  images 
are  hard  to  describe  and  easily  disturbed. 

[Arnheim,  1969,  p.  116] 
There  is  no  doubt  that  the  use  of  mental  imagery  in 
problem-solving  can  be,  and  hopefully  will  be  refined 
through  subsequent  psychological  studies .   Once  the  sub- 
jects are  aware  of  the  role  of  mental  imagery,  their 
reluctance  to  report  it  will  diminish  and  its  management 
will  become  more  efficient. 
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The  second  area  concerned  over-emphasizing  verbal  re- 
porting versus  the  written  reports.   Since  it  is  easier  to 
speak  than  to  write,  some  subjects  initially  tended  to 
"overplay"  their  verbal  responses,  while  "down-playing"  the 
written  protocol.   Even  though  more  accurate  and  complete, 
analysis  and  management  of  the  taped  verbal  protocols 
became  difficult  when  it  could  not  be  compared  to  the 
subject's  written  protocols.   Further,  to  consider  the 
opposite  extreme,  some  subjects  tended  to  shy  away  from 
any  verbal  responses  when  they  discovered  that  they  were 
to  be  tape  recorded.   Therefore,  even  though  tape  record- 
ings are  more  accurate  and  complete  and,  in  some  cases, 
also  provide  historical  reference,  their  value  is  depend- 
ent on  its  application  and  its  economic  feasibility. 

2.   Error  Response  Pattern 

One  of  the  main  goals  of  this  study  was  to  discover  a 
response  error  pattern  for  each  subject.   This  discovery 
would  have  led  to  a  more  individualized  package  of  instruc- 
tion for  the  intended  user.   However  the  results  were  less 
than  dramatic,  showing  irregularities  and  inconsistencies 
in  subject  responses.   For  example,  even  though  test  re- 
sults for  some  subjects  definitely  indicated  that  they 
had  learned  an  incorrect  form  of  the  concept,  obtaining 
an  error  rate  as  high  as  four  out  of  five  possible  errors, 
it  was  not  conclusive.   An  important  question  that  could 
be  asked  from  these  results:   "if  the  incorrect  response 
form  was  definitely  learned,  why  were  not  five  out  of  five 
errors  produced?".   Another  example,  showing  irregularities 
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or  "funny  stuff"  in  subject's  responses  are  shown  in  test 
results  where  the  subject  is  successfully  progressing 
through  the  problem,  but  then  supplies  the  wrong  response. 
The  principal  challenge,  therefore,  lies  in  analyzing  the 
protocol  data  and  answering  the  following  questions: 

(1)  How  may  these  inconsistencies  and  irregulari- 
ties be  explained? 

(2)  Where  do  we  make  the  cut-off  mark  to  deter- 
mine if  a  subject  has  learned  an  incorrect 
response  form:   is  it  at  fifty  percent 
incorrect  responses  for  a  particular  con- 
cept or  dare  we  raise  or  lower  this  mark? 

These  are  difficult  questions.   A  possible  solution  to 

the  first  question  may  best  be  explained  by  Bruner, 

Goodnow  and  Austin: 

.  .  .  strategies  as  employed  by  people  are  not  fixed 
things.   They  alter  with  the  nature  of  the  concept 
being  sought,  with  the  kinds  of  pressures  that  exists 
in  the  situation,  with  the  consequences  of  behavior, 
etc.   And  this  is  of  the  essence.   For  what  is  most 
creative  about  concept-attainment  behavior  is  that  the 
patterning  of  decisions  does  indeed  reflect  the  de- 
mands of  the  situations  in  which  the  person  finds 
himself. 

(Bruner,  Goodnow,  Austin,  1956,  p.  55] 
In  addition,  Feigenbaum  (1963)  pointed  out  that  some- 
times the  process  we  use  to  discover  or  reveal  something, 
produces  incorrect  responses.   The  interesting  question  is 
"how  often"? 

There  have  been  many  theories  on  how  the  human  mind 
processes  a  problem-solving  task.   One  of  these,  which 
seemed  to  be  the  most  plausible,  and  from  the  protocol 


68 


data,  the  one  generally  followed  in  this  study,  stated  that 
the  problem-solving  process  depended  upon  the  presence  of 
certain  previously  learned  rules  (or  in  simpler  cases, 
concepts) .   These  previously  learned  rules  and  concepts 
had  to  be  recalled,  as  a  first  step,  using  instructions,  or 
in  the  case  of  this  experiment — the  upper  bound  definition. 
Whatever  stimuli  was  used  to  recall  the  previously  learned 
rules,  the  externally  applied  guidance  presumably  functions 
to  increase  the  vividness  (or  "availability")  of  some 
previously  learned  entity. 

At  the  next  stage,  the  individual  searched  for  and 
selected  the  recalled  rules  that  were  specifically  relevant 
to  the  stimulus  situation,  and  rejected  those  which  were 
not.   There  is  bound  to  be  some  "noise"  at  this  stage  of 
the  game,  and  the  process  is  one  of  distinguishing  the 
noise  from  the  relevant  signals.   For  example,  if  the  sub- 
ject is  searching  for  the  upper  bound  of  a  set,  the 
geometric  progression  of  the  set  may  be  quite  irrelevant. 
If  he  is  searching  for  the  trigonometric  relationship  for 
"tangent"  at  different  values  for  "pi,"  the  size  of  the 
triangle  may  be  quite  irrelevant.   Conceptualization  of 
these  features  of  the  stimulus  situation  may  have  been  re- 
called; but  they  must  be  distinguished  from  other  concepts 
and  discarded. 

Combining  subordinate  rules  was  the  next  stage  of  the 
problem  solving  process.   There  are  many  ways  in  which 
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recalled  rules  and  concepts  may  be  combined;  the  object  is 
to  find  the  correct  one.   For  example,  consider  set  five: 

Set  5:   {1,  1/2,  1/3,...} 

The  three  dots  (...)  would  indicate  the  rule  that  the  set 
is  infinite.   However,  the  set  also  stresses  consideration 
for  the  rule  when  the  numbers  are  progressing  in  descending 
order.   Presumably,  in  this  stage,  the  rules  were  narrowed 
to  a  relatively  few  likely  combinations ,  and  thus  reduced 
the  time  of  search. 

At  the  next  stage,  the  subject  arrived  at  a  provisional 
rule  which  he  believed  may  solve  the  problem.   This  rule 
was  then  subjected  to  verification  by  carrying  out  the 
operations  it  suggested  with  reference  to  the  stimulus 
objects. 

The  final  stage  that  the  subject  was  found  to  perform 
was  verification.   Here,  the  provisional  rules  were 
checked  by  application  to  the  specific  problem.   Should 
verification  not  work,  the  subject  returned  to  the  task  of 
trying  new  combinations  of  rules.   When  verification  did 
work,  the  solution  had  been  acquired. 

In  summary,  the  problem  solving  process  used  may  be 
thought  of  as  a  linear  sequence  of  operations  that  are 
actively  carried  out  by  the  subjects  according  to  a  system 
of  links  influenced  by  a  set  of  rules  and  laws.   The  in- 
consistencies and  irregularities  of  the  subject's  responses 
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may  be  a  result  of  the  breaking  of  these  links  or  wrong 
applications  of  the  rules.   Mackworth  C1965)  compares  the 
human  mind  to  a  very  efficient  computer,  composed  of  a 
large  number  of  programs  to  handle  all  required  situations 
(with  modifications) .   He  states  that  the  principal 
criteria  of  problem  solving  is  "choosing  correctly  between 
existing  programs."   Incorrect  selection  of  these  programs 
will  provide  incorrect  or  inconsistent  solutions  to  the 
problem- solving  task. 

The  answer  to  the  second  question  must  be  made  on  a 
judgemental  decision,  taking  into  consideration  the 
objectives  of  the  particular  concept.   The  analysis  has 
indicated,  however,  that  if  there  was  less  than  fifty  per- 
cent error  response  rate  for  a  particular  response  error 
form,  a  definite  error  pattern  could  not  be  determined. 

In  conclusion,  the  many  unanswered  questions  for  de- 
fining a  specific  error  response  pattern  must  be  left  for 
the  behavioral  psychologists.   In  this  study,  the  attempt 
was  made  to  discover  a  specific  error  pattern  for  a  par- 
ticular concept,  using  mental  imagery  as  a  tool.   A  few 
explanations  were  provided  for  the  inconsistencies  and 
irregularities  for  subject's  responses,  but  much  remains  to 
be  done  in  this  area  of  response  evaluation. 
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B.   RECOMMENDATIONS  FOR  THE  RESPONSE  EVALUATION  APPROACH 
IN  THE  CAI  ENVIRONMENT 

Both  Abrahms  (1971)  and  a  report  on  "materials  develop- 
ment procedures  and  evaluation,"  by  the  McDonnell  Douglas 
Astronautics  Company  (1980) ,  indicated  that  a  major  short- 
coming in  CAI  development  is  in  the  area  of  response 
evaluation.   They  stressed  a  need  for  a  method  which  would 
guide  the  lesson  writer,  by  defining  more  sophisticated 
rules  by  which  anticipated  responses  to  constructed  re- 
sponses may  be  judged.   The  technique  provided  in  this 
study  seems  to  fulfill  some,  if  not  all,  of  these  require- 
ments . 

The  major  conclusions  drawn  from  this  study  pertain  to 
the  finding  that  the  response  evaluation  approach  can  be 
used  to  develop  CAI  lessons  with  the  individual  student  in 
mind  rather  than  to  provide  an  additional  step  in  the 
already  tedious  writing  of  course  materials  per  se.   Thus, 
the  intent  of  the  recommendations  outlined  below  is  to 
suggest  potential  benefits  to  be  derived  from  broader 
application  of  an  instructional  technology  which  has  been 
shown  to  be  reasonably  effective  in  this  context. 

An  obvious  next  step  would  involve  further  development 
and  implementation  of  the  response  evaluation  approach  in 
the  actual  CAI  course  material  development  process .   The 
area  which  appears  particularly  promising  is  the  identifi- 
cation of  the  principal  incorrect  responses  for  a  particular 
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concept.   The  educator  may  now  define  his  lesson  branching 
structure  with  these  incorrect  response  forms  in  mind. 
Providing  such  system  responses  as : 


(1)  "Are  you  aware  that  plus  infinity  is  not  the 
upper  bound  for  all  sets," 

(2)  "Are  you  aware  that  an  infinite  set  may  have 
an  upper  bound," 

(3)  and  finally,  "Are  you  aware  that  the  upper 
bound  for  a  set  does  not  necessarily  have  to 
be  a  member  of  the  set." 


With  some  modification  and  further  study,  the  response 
error  pattern  process  could  be  implemented  to  provide  for  a 
more  totally  structured  individualized  CAI  system  for  a 
particular  concept. 

Eventually,  one  of  the  most  promising  areas  for  the 
response  evaluation  approach  would  be  to  broaden  the  area 
of  CAI  instruction  to  a  more  diversified  field  of 
knowledge.   For  example,  CAI  lessons  could  be  prepared  for 
fields  which  were  generally  considered  too  complex  due  to 
their  psychological  response  processes  for  their  mastery. 

Finally,  it  is  recommended  that  any  implementation  of 
the  response  evaluation  process  be  coupled  with  the  follow- 
ing guidelines,  proposed  by  Colonel  Joe  Nastasi,  if  the 
CAI  lesson  is  to  be  a  success: 


(1)   CAI  lesson  writers  must  be  senior  instructors 
who  are  experienced  in  their  subject  matter 
areas  and  in  the  course  to  be  developed. 
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(2)  CAI  lesson  writers  should  be  trained  programmed 
instruction  writers. 

(3)  Learning  how  to  write  viable  and  valid  lessons 
takes  considerable  time,  and  constant  turnover 
of  authors  is  disastrous  to  satisfactory 
progress . 

(4)  CAI  lesson  writers  must  be  assigned  full  time 
duty  of  producing  lessons  for  the  CAI  program. 

(5)  Close  and  continuous  liaison  must  be  main- 
tained between  the  course,  other  instructors, 
CAI  lesson  writers,  the  educational  con- 
sultant and  the  CAI  project  operations. 

(6)  An  authorized  course  subject  block  diagram, 
with  indicated  lessons,  examination  questions, 
behavioral  objectives,  plan  of  each  lesson, 
and  lesson  flowcharts  must  be  prepared  prior 
to  writing  and  coding  lessons ,  and  be  avail- 
able to  insure  valid  CAI  lesson  development. 

(7)  The  officer  in  charge  and  chief  instructor  of 
the  course  being  developed  must  retain  full 
control  and  supervision  of  the  CAI  lesson 
writers  and  their  efforts . 

(8)  All  CAI  lessons  must  be  very  carefully  re- 
viewed from  a  pedagogical  and  student  view- 
point. 


[Prokop,    1976,    p.    218-219] 


74 


APPENDIX  A 
TEST  DATA  FROM  SUBJECTS 


SI  CD 


SI  CD. 


DEFINITION ;       Let    E    be    a    subset    of  the  real  numbers,   R.     An  element, 
m,   of  R  is   said  to  be  cui  upper  bound  of  E  if  x  is  equal  to  or  less  than 
m  for  every  x  in  E. 

1.  e£r 

2.  m£  R 

3 .  V  X  —  m  ^^  m  =  iipper  bound . 

PRACTICE  PROBLEM;   Let  E  be  the  set  ^1,   2,    3,  4  j ,  i.e.  the  set  consisting 
of-  the  first  four  positive  integers.  Does  E  have  an  upper  bound? 

1.  See  four  finite  elements. 

2.  Looked  for  greatest  value. 

3.  Yes,  has  upper  bound  of  4. 

SET  2:   The  set  of  natural  numbers,  N  (i.e.  the  positive  integers) 
1.   I  see  an  infinite  set. 


2.  An  infinite  set  is  unbounded. 

3.  No. 

SET  4:   A  finite  set. 


1.   No  mental  image  at  first.   Thought  about  it, 
Then  saw  100  tanks. 

1  '2   3       100 
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SI  CD. 


2.   Yes,  there  is  upper  bound  of  greatest  element  in  set, 

SET  5;    1,  1/2,  1/3,  ... 

1.   I  see  infinitely  smaller  numbers. 


2.   Therefore  upper  bound  must  be  the  greatest  number  in  the  set. 
SET  6;  The  set  of  real  numbers,  R. 

1.  No. 

2 .  An  infinite  set    ( ni^xonboxinded)  . 

SET  11;  The  non-negative  numbers. 

1.  Infinite  set  starting  at  0. 

^ 

2.  No,  same  as  set  6. 

SET  13;   The  closed  interval  I  3,  5 J. 
1.  Yes. 


2.  The  largest  element  in  set  is  upper  bound. 

3.  Finite  set. 


SET 


r? :  S^>   ^'  ^,  ^,  ^,  ^.  ...  3 
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SI   CD. 


2.      after  test 
No. 


SET  18;     fTan  x,   where  x  =/7^   2Tf,    3/^    4V^   5//,   6^    •••J 

1.  Yes.   (guess  since  tangent  is  bounded  by  0  ,  1). 

2.  I  see  an  set  of  O's  and  I's 


;  [l,  2,  4,  8,  16,  ...  } 


SET  21 


1.  I  see  numbers  increasing. 

2.  No,  infinite  set. 
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S2  L.M. 

S2  L.M. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  boxand  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 

1.   Definition  thought: 

The  real  nimibers  ordered  on  a  number  line. 

-3-2-10123 
4—1 1 1 1 1 1 f — > 


PRACTICE  PROBLEM;   Let  E  be  the  Set  £_1,  2,  3,  4  J,  i.e.  the  set  consisting 
of  the  first  positive  integers.  Does  E  have  an  upper  bound? 


1.  12   3  4 

H \ 1 K? 

2.  upper  bound  =  4. 

SET  2;  The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers) 


1.       1   2   3   ... 

I 1 \ ^ 


2.   No  bound,  it  goes  to  infinity. 


SET  4:   A  finite  set. 


1.   Points  on  the  line  —  bounded  by  the  element  which  falls  farthest 
to  the  right  on  the  number  line   Clike  definition! . 


SET 


±1    ^1,  1/2,  1/3,  ...] 


1.   I  see  an  infinite  number  of  points  getting  smaller, 
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2.  0  1  ^ 

<H • • 1— ^ 

1/3   1/2 

3.  bovmded  by  0. 

4.  I  remember  definition  and  it  asks  for  upper  boiand,  therefore -ss^  1. 

SET  6;  The  set  of  real  numbers,  R. 

1.  no  bound. 

2.  • ^ 

SET  11;   The  non-negative  numbers. 

1.  I  see  an  infinite  set. 

2.  Upper  bound >0^  (the  first  non-negative  number). 

3.  ...   -2  -1  0 
< 1 1 4 

4.  Qjo  lower  bound  J 

SET  13;  The  closed  interval  Q,  5  J. 
1.  Saw  the  largest  nianber  5. 


2. 

3 

r 

1 

5 
1 

L 

1 

J 

3. 

bound 

5. 

SET 

17:     ^ 

■^, 

V3, 

V4, 

,  V5 

1.  No  bound  —  each  number  bigger  than  the  previous  number, 
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SET  18;  ^  Tan  x,  where  x  =  ^,2^,37', 4'^, 5^,6 ;r,  ...2 

1.  I  could  not  remember  definition  of  tan. 

2.  I  see  ...  ,  therefore  goes  to  infinity. 

3.  No  bound. 


SET  21: 


^1,  2,    A,    Q,    16,  ...  j 


1.      Increasing  infinite  set. 

I 


12      4      8 


2.      no  bound. 


80 


S3  T.V. 


S3  T.V. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.   xi  m  for  every  x  in  E. 

PRACTICE  PROBLEM;  Let  E  be  the  set£l,  2,  3,  4 J  ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 

1.  Yes,  4. 

2.  Four  is  the  largest  number  and  so  all  of  the  others  are  less  than 

or  equal  to  it  so  it  must  be  the  upper  bound. 

SET  2;  The  set  of  natural  nuinbers,  N  (i.e.  the  positive  integers). 

1.  No  upper  bound  —^infinite. 

2.  Imagery —  (1,  2,  3,  ...> 

SET  4;   A  finite  set. 

1.  I  see  a  certain  niomber  of  points. 

2.  Imagery  —  o^'  ^'    ^'   ^  J 

3.  Upper  bound  equals  largest  number  in  set. 


SET  5 


:   £l,  1/2,  1/3,  ...^ 


1.  1  is  upper  bound. 

2.  I  visualize  the  numbers  trailing  off  to  right  so  1  is  largest. 

My  first  impression  was  that  it  was  an  infinite  set  with  no 
upper  bound. 
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SET  6 ;   The  set  of  real  numbers,  R. 

1.  Infinite  set,  no  upper  boxond. 

2.  Imagery      ^1,    2,    3,    4,    ...  i, 

SET  11;  The  non-negative  numbers. 

1.  Infinite  set. 

2.  no  upper  bound   (see  abovel. 


SET  13:  The  closed  interval 


[3.  5]. 


1.  5  is  the  upper  bound. 

2.  It  is  the  largest  nvimber  in  a  finite  set. 

SET  17;  £V2,V§',V4,V5,V6,V7^  .  . .  T 

1.  Infinite  set  —  no  upper  bound. 

2.  I  first  look  at  the  values  of  the  perfect  squares. 

3.  Imagery 

I 1 1 =P 

12      3 

SET  18;   f  Tan  X,  where  x  = /T",  2/7,  377^  4^,  5/^  6/7;  ...1 

1.  Visxialize  coordinate  plot  of  trig  values. 

2.  tan  =  sin 

cos 


82 


S3  T.V. 


3.   Infinite  set  —  no  upper  bounds, 
,:   ^  1,  2,    A,    8,  16,  ...J 


SET  21; 


1.  Infinite  set  no  upper  bound. 

2 

2.  Imagery  =  X 
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S4  R.W. 

DEFINITION ;  Let  E  be  a  subset  of  the  real  niiinbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 

1.   DEF:   For  all  answers  dO   is  no  bound 

PRACTICE  PROBLEM;  Let  E  be  the  set/1,  2,    3,  4  J  ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 

1.  E  has  an  upper  bound. 

2.  m  =  4,  because  set  E  has  ascending  order. 

3.  12   3  4 

I i 1 1 

SET  2;   The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers) . 


1.  1  2   3   4   ...  +oO 
I 1 1 1 — 

2.  There  is  no  upper  bound,  unlimited. 


SET  4;   A  finite  set. 

1.  A  finite  set  not  necessarily  has  an  upper  bound,  becaxise  "finite" 

is  concerned  with  the  number  of  elements  in  the  set. 

2.  After  asking  examiner  —  this  set  is  bounded  above. 
SET  5;  ^1,  1/2,  1/3,  ...  3 

1.  1        0 

I M 4-t—^ 

2.  The  upper  bound  of  this  set  is  1,  because  the  set  has  a  descending 

order  in  its  elements. 
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3.      01=1 


SET  6;        The  set  of  real  nxoinbers,   R. 


1.        OO  - 


+  CO 


2.   The  set  of  real  niombers  has  no  upper  bound  because  they  go  to  + 


SET  11;   The  non-negative  numbers. 


1. 


I    ^i.»n»  <-*-r->.  . 


2.   No  upper  limit,  goes  to  cO 


SET 


13;  The  closed  interval  [_3,  5j. 


1.  [3,  5j 

2.  The  closed  interval  has  an  upper  bound. 

3.  m  =  5. 

SET_17;  fV2,\/3,V<4,V^,V6,N/7,  ,..\ 


1. 


V5  v?  v^   . 

4 1 » ^ 


•  oo 


2 .  no  upper  bound 


SET  18;  j   Tan  x,  where  x  =7r ,  2T,    3u',   4",   5^^,   6^  •••  } 


:    I         I  / 


/^ 
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2.  Set  contains  all  zeroes,  E^O,  Q,  0.,    ...  1 

3.  Set  has  no  upper  bound. 


SET  21; 


(l,  2,    4,  8,  16,  ...J 


1.  Set  has  no  upper  bound. 

2.  Set  of  the  squares  of  natural  numbers   ^r  oO 
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S5  S.L. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  nximbers,  R.   An  element,  m 
of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 
m  for  every  x  in  E. 

1. 


2.  m^(E] 

3.  X  =[e]'—  m 

4.  m  =  element  in  E, 


PRACTICE  PROBLEM;   Let  E  be  the  set  {l,  2,    3,  4},  i.e.  the  set  consist- 
ing of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  [eJ  =   (1,  2,  3,  4) 

2.  m  =  4 

SET  2;   The  set  of  natural  niombers,  N  (i.e.  the  positive  integers) 

1.  Natural  numbers,  N,  5 unlimited  positive  number. 

2.  So  there  should  not  be  any  upper  bound. 

SET  4:   A  finite  set. 

1.  A  finite  set  — >  the  set  is  finite. 

2.  So  should  be  an  upper  bound  of  the  subset  E. 

SET  5:  {1,  1/2,  1/3,  ...  } 
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1.  CI,  1/2,  1/3,  ...1 ^  unlimited. 

2 .  Fraction j^  1/n . 

3.  1  =  the  upper  bound. 

SET  6;   The  set  of  real  numbers,  R. 
1.   Undefined  property   ^ 


2.  The  set  of  real  numbers,  R,_^if  it  means  all  real  nimibers _^then 

there  is  no  upper  bound. 

3.  If  it  starts  from  a  certain  number  downward y  there  is  an  upper 

limit. 


SET  11;   The  non-negative  numbers. 

1.  All  non-negative  numbers. 

2.  No  upper  bound  except 


SET  13:   The  closed  interval 


1.    I f 

3       5 


[3,  5j. 


2.   5  =  upper  bound. 

SET  17;  ^v^,V3,\/4,\/5,V§y7,  ...J 

1.  (V5  ;v^ ,  ...  ,>/7 ,  . . .  ^  =^  Vn 

2.  -^       no  upper  bound. 

SET  18;  ^Tan  X,  where  ^  =7r,    2lf,    2^,   A^,    5/7^  STf,    ...  V 
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Tan/^=  Tan  2  ^ 


2 .      upper  bound  =  0 


S5  S.L. 


SET  21 


:    £  1,  2,    4,  8,  16,  ...J 


1.  See  an  infinite  increasing  set. 

2.  no  upper  bound  — ^  2^  . 
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S6  P.H. 

DEFINITION ;  Let  E  be  a  subset  of  the  real  numbers,  R.  An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  mC  R. 


PRACTICE  PROBLEM: 


Let  E  be  the  set^l,  2,  3,  4 J-  ,  i.e.  the  set  consisting 


of  the  first  four  positive  integers .   Does  E  have  an  upper  bound? 

1.  Yes 

2.  upper  bound  —  4. 

SET  2 ;   The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers!. 

1.  N  =  [l,  2,  3,  ...  J 

2 .  Yes ,  j>  always  a  real  number  that  greater  or  equal  to  the  largest 

integer . 

SET  4;   A  finite  set. 

1 .  A  =  ^  al ,  a2 ,  . . . ,  aiX 

where  i  is  finite  number 

2.  yes  bounded. 


SET  5 


:   {l,  1/2,  1/3,  ...j 


1.  The  set  decreases. 

2.  Yes,  bounded  =  1. 
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SET  6;   The  set  of  real  numbers,  R. 


1.   <i 
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2.  No,  real  nuuonbers  not  bounded  upper  side. 
SET  11;  The  non-negative  numbers. 

1.  No 

2.  Non-negative  numbers  could  be  real  — p  so  the  same  as  set  6  above, 


SET 


13;   The  closed  interval  |  3,  5 J 


1.      3 


2.   Yes,  bound  ^  5. 


SET  17;  ^N/2,V3,N4,V5,V6,V7,  ...j 


Yes,  because  the  number  under  square  root  are  integers;  we  always 
could  find  a  real  number  that  is  larger  than  that. 


s/2    s/l    y/i 
I ' ^■ 


Vn 

— H- 


SET  18;  V  Tan  x,  where  x  =  ■^,  IT',   2%   4^,  ST,   er,    . .  ,X 


1.   ^ 


2.  always  tan  x  =  0 

3.  yes,  an  upper  bound 
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SET 


21:  j  1,    2,    4,  8,  16,  ...  2 


1.  Set  of  integers. 

2.  Yes  would  be  upper  bound. 

3.  Same  as  set  2. 
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S7  Y.K. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  number,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  eqxoal  to  or  less  than 

m  for  every  x  in  E. 

PRACTICE  PROBLEM;   Let  E  be  the  set  >  1 ,  2,    3,  4j,    i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  an  upper  boxand? 


1.  Yes,   the  upper  bound  is  4. 

2.  Each  number  in  the  set  is  less  than  4. 

SET  2 ;  The  set  of  natxaral  numbers,  N  (i.e.  the  positive  integers) 

1.  Yes 

2.  There  is  a  number  greater  among  the  other  numbers. 

3.  Examiner  questions  answer: 

4.  Infinity  is  the  bound. 

SET  4:   A  finite  set. 


1.   Yes,  same  as  set  2. 

[l,  1/2,  1/3,  ...j 


SET  5: 


1.  yes 

2.  The  upper  bound  is  1 

3.  Each  number  in  the  set  is  less  than  1, 

SET  6;   The  set  of  real  numbers,  R. 

1.   Yes,  same  as  set  2. 
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SET  11;   The  non-negative  numbers. 
1.   Yes,  same  as  set  2. 

[3,5] 


SET  13:   The  closed  interval 


1.  Yes. 

2.  5  is  the  upper  bound. 

SET  17;  ^\/2,\/3,N/4,\/5,Vi,V^,  ...J 

1.   No,  because  it  is  infinite. 

SET  18 ;  f  Tan  x,  where  x  =77,  2/7^  3T,   4%   5"^,  6T,  . .  .1 

1.  No  upper  bound 

2.  Goes  to  infinity 

SET  21;  £1,  2,  4,  8,  16,  . . .| 

1.  No  because  it  is  infinite. 
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S8  R.F. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  eq\ial  to  or  less  than 

m  for  every  x  in  E. 


subset  E  of  real  nxanbers,   R. 


X  ^  m 


m  is  only  an  upper  bound  of  subset  E  if  x  —  m 


PRACTICE  PROBLEM;   Let  E  be  the  set  Tl,  2,   3,   4y  ,   i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  E  has  an  upper  bound  (.41 

2.  Reason  is  that  it  is  a  finite  set,  so  the  highest  number  is  the  upper 

boxind. 


3.   E  =  £l,  2,  3,  4^ 


SET  2 ;   The  set  of  natural  numbers,  N  (i.e.  the  positive  integers 1 

l.Sl,  2,  3,  ...,  n^ 

2.  infinite  set 

3.  no  upper  bound 
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SET  4;   A  finite  set 

1.  ^1,  2,    3,  4,  Sj 

2.  has  upper  bound  — ^  in  my  ex.  upper  bound  is  5, 


SET  5 


:  ^1,  1/2,  1/3,  ...j 


1.  decreasing  set  to  infinity. 

2.  def.  of  upper  bound  means  the  highest  number  in  that  set. 

3.  The  upper  bound,  in  ex.,  is  the  highest  number  Cll. 

SET  6:  The  set  of  real  numbers,  R. 

1.  Does  not  specify  what  R  contains. 

2.  It  is  an  infinite  set. 

3.  Correction  —  it  does  specify  what  R  contains  —  all  real  numbers 

4.  All  real  numbers  is  an  infinite  set. 

5.  Thus  no  upper  boiond. 

SET  11;   The  non-negative  numbers. 

1.  Means  all  positive  numbers. 

2.  All  positive  numbers  is  an  infinite  set. 

3.  No  upper  bound. 


SET  13:   The  closed  interval 


[3,5] 


1.      Contains  only  numbers   3  and  5. 
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2.  Finite  set 

3.  Upper  bound  is  5. 

SET  17;  fV2A/3,V4,>/5,V6y7,  ...J 

1.  Increasing  set. 

2.  Increasing  by  one  square  root  digit. 

3.  In  order,  so  that  the  next  number  will  be 

Vs,  Vs,  v/io,  ...  ,\/n  J 

4.  It  is  an  infinite  set. 

5.  Does  not  have  an  upper  boiond,  except  for  the\/h,  which,  is  tondefined 

at  present. 

6.  I  would  say  this  set  does  not  have  an  upper  boimd. 
SET  18;  £  Tan  x,  where  x=7^,    2T,  37r,   4T,    STt,   ^TT,    ...1 

1.  Increasing  set. 

2.  Same  as  problem  set  17. 

3.  No  upper  bound. 


SET  21 


;  {l,  2,   A,    8,  16,  ...} 


1.  increasing  set  in  sequence  of  2^ 

2.  infinite  set. 

3.  No  upper  bound. 


97 


Sa  F.K. 


S9  F.K. 

DEFINITION ;  Let  E  be  a  sxobset  of  the  real  nxombers,  R.   An  element, 

m,  of  R  is  said  to  be  an   upper  boiond  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  I've  seen  something  like  this  before, 

2.  /xl,   x2,   x3,   x4\ 
\  1    2    3    4] 


3. 


4.   xi  -^  m. 


PRACTICE  PROBLEM;   Let  E  be  the  setjl,  2,   3,   4,j  ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  an  upper  bound? 


1. 
2. 

3. 

4. 


I  see  a  finite  set. 

12   3  4 
• •   »   • 


According  to  definition  xi^  m 
Yes,  upper  bound  =  4  or  greater. 


SET  2;  The  set  of  natural  numbers,  N   (i.e.  the  positive  integers). 


1.  what  is  a  natural  number  —  is  0  a  natural  nxmnber? 


2.   This  set  goes  to  infinity 


• — I 1 

0   12... 


00 
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3.   Since  infinity  is  the  largest  number  —  the  upper  bound  is 
infinity . 


SET  4;  A  finite  set. 

1.  a[i,  2,    2,    A,    sj- 

2.  This  is  just  like  the  practice  problem. 

3.  Upper  bound. 


I     SET  5;  £l,  1/2,  1/3,  ...J 


1.   I  see  the  set  decreasing. 
1/4,  1/5,  1/6  =>  1/n 

3.  Since  it  is  decreasing  the  largest  number  must  be  the  upper  bovind. 

4.  Yes,  upper  bound  =  1. 

SET  6;   The  set  of  real  nxombers ,  R. 

1.  what  is  a  real  number? 

2.  Rational  numbers  such  as  -3/4,  18/29-,  3,  -6 

3.  Irrational  numbers  such  as  N/2   \3/5,  N.  5/  -3 

4.  It  (real)  goes  to  infinity. 

5.  Infinity  must  be  the  upper  bound,  since  there  is  no  number  greater 

than  infinity. 

SET  11;  The  non-negative  numbers. 

1.   I  see  a  line  going  towards  the  left  to  - cP 


^ < *- 
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2.  Therefore  zero  must  be  the  upper  bound. 

3.  Wait,   it  wants  non-negative  nuinbers. 

4 .  The  line  goes  to  the  right  to  +  oo 

5.  Since  +   is  the  largest  possible  number,  it  must  be  the  upper 

boxond. 


SET  13;  The  closed  interval 
1.   I  see  a  line  like  this 

b i 


[3,5j 


2.  From  previous  e3q)erience  this  is  a  finite  set,   with  boundaries 

3  and  5 

3.  upper  bound  of  5. 

SET  17 ;  ^V2  ,V3  ,V4  ,V§  ,\/6  }/l ,  ...J 

1.  I  first  see  an  infinite  set. 

•  •  • 

2.  I  then  look  at  the  "\/~" 

3.  I  ignore  this,  since  infinity  suggests  cin  upper  bound. 

4.  Yes,  upper  bounded  above. 

SET  18;   £Tan  x,  wherex  =  TT,   2%  zK  ^K  5^,   ^%    ..."l 

1.  I  see  an  infinite  set. 

2.  I  immediately  say,  yes  bounded  above. 

3.  I  look  at  what  tangent  means 

sin 

tan  = 

cos 
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4 .  what  are  the  values  for  tan  at  If ,      2  ^,  etc , 

5.  I  draw  sine  curve  and  cosine  curve. 


OoQc< 


6.  Determined  that  tan  x.  =  S  Q,0,0,...  c 

7.  Upper  bound  of  0  or  greater. 


SET  21 


:  ^1,  2,    A,    8,  16,  ...j 


1.   I  see  a  progression  —  2^ 


2.   It  goes  on  —  no  upper  bound, 
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SIO  D.M. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  niimbers,  R.  An  element, 

m,  of  R  is  said  to  be  an  upper  boxond  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1. 


2.   x^m^^R 

PRACTICE  PROBLEM;   Let  E  be  the  set  [l,   2,   3,   4J  ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  This  problem  is  obvious. 

2.  12   3  4 

i t—t ♦ 

3.  it  is  finite,  therefore  upper  bound  of  4, 


SET  2 :  The  set  of  natural  numbers,  N   (i.e.  the  positive  integers). 


1.   I  immediately  see  an  infinite  set  going  towards  plus  infinity, 

« -> 


2.   This  implies  to  me  that  no  upper  botind  exists, 


SET  4:   A  finite  set. 


1.   Obvious,  no  upper  bound. 


SET  5 


{l,  1/2,  1/3,  ...  j. 
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1.  I  see  an  infinite  set. 

2 .  It  decreases  to  zero  —^therefore ,  the  largest  nvunber  must  he  the 

upper  bound. 

3.  Almost  tricked  me  by  showing  infimite  set. 
SET  6;  The  set  of  real  numbers,  R. 

1.  I  immediately  see  an  infinite  set,  which  causes  me  to  say  no  upper 

bound. 

2.  I  see  no  other  images. 

SET  11;   The  non-negative  numbers. 

1.  I  draw  a  picture 

^ -, 

2.  Since  0  is  the  first  non-negative  number,  it  must  be  the  upper  bound. 


SET  13:   The  closed  interval 


r_3,  =3 


1.  There  are  an  infinite  number  of  points  between  3  and  5,  but  endpoints 

are  3  and  5. 

3      5 

C 1 

2.  Yes,  upper  bovind. 

SET  17;  /Vf  ,\/3 ,  V4  ,\^  ,\/6  ,N/7  ,  . .  .  | 

1.  I  see  an  infinite  set. 

2 .  I   ignore  the      "  V~ 
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3.  Set  goes  to  infinity 

4.  No  upper  bound 


SET  18;  ^  Tan  x,  where  x  =77",  27f,   37f,   4/7)  sP',   6/7)  . .  .^ 


1.   Tan  X  =  x/y 


2. 


^  r 


y 


3.  whenever  x  =  0,  y  is  either  1  or  -1 

4.  therefore  tan  x  =4o,  0-,  0,  •••7 

5.  Set  has  an  upper  bound 


SET  21 


.  j    1,    2,    4 ,  8 ,  16 ,  . . •  1 


1.   I  see  an  infinite  set. 


2.   Progression  =  2 


n 
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sii  s.w. 

DEFINITION ;   Let  E  be  a  stibset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.   X-^E_^ 
2. 


3.  m  _;^R 

4.  X  ^  m 


PRACTICE  PROBLEM;   Let  E  be  the  set  |  1,  2,  3,  4  J- f  i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  an  upper  bound? 


1. 


P,2,3,J) 


2.  Since  E  is  a  finite  set,  there  exists  a  R  greater  than  E. 

3.  Upper  boxind. 

SET  2;  The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers ]l. 


1.  No  real  images. 

2.  Since  natural  numbers  go  on  indefinitily,  it  implies  an  infinite 

set. 


3.  No  upper  bound. 
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SET  4;    A  finite  set. 

1.  How  can  I  assign  an  m? 

2.  I  don't  see  any  numbers  or  elements. 


SET 


_5:  {l,  1/2,  1/3,  ...J 


1.  I  see 

•  •  • 

2.  Infinite  set  has  no  boiind,  but  wait  it  is  decreasing, 

3.  I  draw  picture 

<— . i 1 

1/3    1/2    1 

4.  The  progression  is  1/n  _^  goes  to  0 

5.  Upper  bound  is  the  largest  nxomber  —  1. 

SET  6;   The  set  of  real  numbers,  R. 

1.  I  see  this  immediately  as  an   infinite  set. 

Therefore  not  bounded. 

2.  No  mental  pictvires  appeared. 
SET  11;  The  non-negative  numbers. 

1.   Draw  picture  of  the  non-negative  numbers. 


0  +oo 

2.  Goes  to  infinity. 

3.  No  upper  bound. 
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SET  13:   The  closed  inteirval 


[3, 5] 


1.  First  looked  at  bracketed  numbers  and  recognized  in=5. 

2.  Next  looked  at  the  word  interval  this  iirplied  infinite  set. 

3 .  Since  it  is  bracketed  m  S  5 . 

SET  17;^S/2,\/3,V^y5,V6,\/^,  . . .  } 

1.  Saw  numbers  with  square  root  signs. 

2.  Saw  dots  which  implied  infinity  and  no  way  of  assigning  an  m. 

3.  numbers  are  increasing  under  square  root  signs. 

4.  infinite  set,  m  can  not  be  defined. 

SET  18 ;{  Tan  x,  where  x  ^Tf ,   2%   ZTf,    ATf,   5^  6^  ,..^ 

1.  First  saw  an  infinite  set. 

2.  The  value  If   for  tan  looks  familiar. 

3.  Draw  picture 


4.  tan  X  =  sin  x  -,  TT  =■  0       2  77*  =  0. 

cos  X 

5 .  the  set  is  /  0 ,  0 ,  0 ,  • • •  V  - 

6.  upper  bound. 

_:  /  1,  2,    4,  8,  16,  ...  ^ 


SET  21: 


1.   infinite  set  came  to  mind  first. 
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2.      Same  as   set   17. 
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S12  B.N. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.  An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  First  time  I  read  definition  it  didn't  make  sense. 

2.  e€  R 

3.  m€   R 

4.  X  C  E 

5.  If  all  of  the  above  true,  then 

X  ^  m 
for  upper  botind  to  be  true. 

6.  Checking  to  see  if  that  makes  sense. 

PRACTICE  PROBLEM;   Let  E  be  the  set  |^1,  2,  3,  4,^,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  iipper  bound? 

1.  Instantaneous  reaction. 

2.  Verify  upper  boiind,  also  lower  bovmd. 

3.  Yes,   finite  and  bounded  in  both  direction 

SET  2;  The  set  of  natxiral  numbers,  N  (i.e.  the  positive  integers  1_. 
1.   I  see  an  infinite  line  going  to  the  right. 


2.  I  remember  definition  said  real  members 

3.  Since  infinity  of  real  numbers  not  same  as  infinity  of  natijral 

numbers. 
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4.   The  set  is  boiinded  by  infinity  of  real  nombers 
SET  4;   A  finite  set. 

1.   It  has  boxond  (from  memory,  practice  problem!.. 
_:  £  1,    1/2,  1/3,  ...  1 


SET  5; 


1.  see  infinite  set. 

2 .  inspect 

3.  Find  pattern — ^1/n. 

4.  Get  smaller  to  right. 

5.  /^   Find  largest  number. 

6.  Boionded  by  1. 

SET  6;   The  set  of  real  numbers,  R. 

1.  This  is  just  like  set  2. 

2 .  Is  there  a  number  —  infinity  for  real , 

3.  No,  /•  boxanded  by  infinity. 

SET  11;   The  non -negative  numbers. 

1.  These  are  natural  numbers. 

2.  This  type  of  question  was  already  asked. 

3.  Is  this  a  trick?? 

4.  It  goes  to  infinity/^  bounded  by  infinity  of  real, 
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SET  13:   The  closed  interval  \_3 ,   sj 

1.  I  see  3  and  5. 

2.  5  is  largest. 

3.  upper  bound. 

SET  17 ;  f \/2  y3  ,V4  >/5  ,\/6  >/7 ,  . .  .X 

1.  This  is  an  infinite  set  from  dots. 

2.  Sequence  >yfi" 

3.  Can  I  find  m  larger  than  Vn 

4.  I  can't  see  m 

5.  not  botanded. 

SET  18;  £  Tan  x,  where  x  =T,   277",  3^  4/7",   ST,   eT,    . .  .\ 

1.  Compute  tangent 

sin  X 

tan  X  =  

cos  X 

Put  in  the  77"  's  for  the  ^'s 

2.  cos  7T      1  or  -1;      sinTT— ?  0 

3.  Constant  0. 

4.  mi  0     .*.    bounded. 


SET  21 


:     {l,    2,    4,    8,    16,    ...J 


1.  Numbers  seem  to  be  increasing  in  some  order. 
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2.   I  think  N^ 


3.  See  dots  /#  infinity 

4.  Can  I  find  m? 

5.  No. 
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S13  E.G. 

DEFINITION;   Let  E  be  a  siibset  of  the  real  nvnnbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  Definition  is  familiar. 

2.  Vx,  x£  E,  X  —  m 

3.  E  ^  R 

4.  If  true,  upper  bound. 

PRAGTIGE  PROBLEM;  Let  E  be  the  set  £  1,  2,   3,   A  J  ,   i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  finite  set 

2.  xi  m 

X  =  1,  2,  3,  4 

3.  m2  4 

4 .  Bounded 

SET  2 ;  The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers). 

1.  Natural  numbers  do  not  include  irrational  numbers 

2.  Horizontal  in  space. 

3.  Unbounded  to  right. 

4.  Bounded  on  left. 

SET  4;      A  finite   set. 

1.      Biggest  element      CautomaticI 
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SET 


2:  {  1,  1/2,  1/3,  ...  j. 


1.  1/n  first  thought. 

2.  infinite  decreasing 


, 1 1 

1/3   1/2   1 


3.  If  n  very  large  sequence  goes  to  0. 

4.  1  biggest. 

SET  6;  The  set  of  real   numbers,  R. 

1.  line  appeared  — ^  continuous 

2.  Continuous  -r^^  infinity ,'  no  upper  or  lower  bound. 


SET  11;      The  non-negative  nxjmbers. 


1.  0,    1,    2,    3,    ... 

2 .  Same  as  above :   continuous  ^^  infinity  « •  no  upper  but  h^s  lower 

bo\jnd. 


SET  13:   The  closed  interval 


C3,  5]. 


1.  interval  ^:>  infinite  number  of  points 

2 .  biggest  is  5  /#  m  -21  5 

SET  17;  j'>/2y3,N/4,V5y6,V^,  ...] 
1.   Noticed  integers  as  written. 
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2.  Integers  unbounded. 

3.  Replaced  V 

4.  Still  increasing. 

5.  Above  set  unbounded. 

SET  18;    {.Tan  x,   where  x  =77,   2T,   ZV,   Ajf,    5^    STT,    . .  .y 

1.  tanT=  0 
tan  2T    =  0 

2.  Are  all  elements  the  same? 

3.  Look  at  graph  vs.  computed  values 


ffr 


4.  The  elements  are  the  same 

5 .  must  be  bounded  21  0 

SET  21;  ^  1,    2,    4,  8,  16,  •••'l 
1.   infinite  set 


2.   Progression  2^  has  no  real  value. 


3.   unbounded. 
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S14  G.J. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  nvanbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  What  are  the  relationships  between  Cx,  m,  E,  R)? 

2.  EC  R 
xC  E 

mC  R 
x-S  m 

3.  I  had  to  read  definition  several  times. 
4. 


PRACTICE  PROBLEM;   Let  E  be  the  set)  1,  2,   3,   4  I ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  cm  upper  bound? 


1.   Yes,  the  upper  bound  is  4. 

This  is  an  immediate  reaction,  since  the  set  is  not  infinite. 

SET  2 ;   The  set  of  natural  nixmbers,  N  (i.e.  the  positive  integers! 

1.  No. 

2.  I  picture  mental  image  of  above  set,  but  continuing 

3.  Can't  be  upper  bound<—> infinite. 
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SET  4;   A  finite  set. 

1.  What  is  the  definition  of  a  finite  set? 

2.  Definition  states  that  an  upper  bound  and  lower  boxjnd  exists. 

images : 

1,  2,  3,  4 

3.  Therefore  has  bound. 


SET  5 


:  {l,  1/2,  1/3,  ...j 


step  1.   I  see  this  set  as  a  decreasing  fraction,  therefore  I  tentatively 
conclude  that  the  values  will  continue  to  decrease. 

step  2.   Notice  a  pattern  in  the  set.   Numerator  is  constant,  denominator 
is  increased  by  1. 

1.  Discovered  that  set— ^0 

2.  This  must  be  the  lower  bound. 

3.  Examine  set  for  greatest  value. 

4.  m  =  1. 

SET  6 ;   The  set  of  real  numbers,  R. 

1.  Immediately,  I  remember  the  definition. 

2.  Conclude  that  there  is  no  upper  bound. 

SET  11;  The  non -negative  numbers. 
1.   Define  non -negative 
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2.  This  has  same  property  as  the  set  of  natural  numbers. 

3.  State  the  same  conclusion  that  there  is  no  upper  bound. 


SET  13:   The  closed  interval 


L3,S] 


1.  Examine  the  interval. 

2.  State  its  properties:   3—  x  —  5 

3.  State  conclusion:  m  =  5 
Image : 


^  <r—^\ 


SET  17;  lyi ,\/3,s/4y5 ^l yi ,  .  . .  } 

1.  Change  squares  into  familiar  digits 

\/\     =   1.414  s/z     =  1.732   \/4  =  2   etc. 

2.  I  notice  the  values  increasing  without  bound. 

3.  Out  of  habit,  there  is  no  upper  bound. 

4.  I  was  busy  with  the  calculations  so  I  did  not  see  any  images. 

SET  18;  -{^Tan  x,  where  x  =F,  2T,   3^  4^  577^  67^  •••  V 

1.  Define  fxmction  in  more  familiar  terms. 

tan  X  =  sin  x/  cos  x  =  0 

2.  Above  expres-sion  =  0  always,  since  we  deal  with  angles  where 

numerator  is  always  0.. 

3.  Conclude  that  the  function  neither  increases  or  decreases. 

4.  The  upper  and  lower  bound  =  0. 
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SET  21 1 


^    1,    2,    4,  8,  16,  ...  J 


1.  Notice  the  numbers  increase  initially. 

2.  Confirm  that  this  will  always  increase  without  bovmd. 

3.  Conclude  that  there  is  no  upper  boxind. 

No  imagery.   Strictly  computational. 
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S15  R.C. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  R. 


1.  m  must  be  to  the  right  of  E  Cthe  elements  of  E) . 

E 

2.  Note  that  xi  m 

PRACTICE  PROBLEM;   Let  E  be  the  set  £l,  2f   3,  4^  ,  i.e.  the  set  consisting 
of  the  first  four  integers.   Does  E  have  cin  upper  bound? 

1.  Yes. 

2.  m  =  4 

3.  Re-read  definition  to  see  if  upper  bound  could  be  equal  to  an  element 

of  E. 

SET  2;  The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers!. 

1.   No. 

I  realize  set  was  infinite. 

SET  4.   A  finite  set. 

1.  Yes  m  =  largest  element. 

2.  I  visualize  a  finite  set  as  a  set  of  points 

.   #   f   # 

3.  Took  largest  point  as  the  upper  boiond. 
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SET 


5_:  [l,    1/2,    1/3,    ...  ^ 


yes,   in  =   1. 

Visxialization  of  1  cuid  1/2  and  1/3. 

Realized  points  were  going  off  to  the  left. 

^ 

1/3   1/2    1 

Since  we  are  interested  in  the  values  on  the  right 

m  =  1. 

SET  6;  The  set  of  real  numbers,  R. 

1.  No. 

2.  Automatically  realized  set  had  no  upper  bound. 

No  image  of  R. 

SET  11;   The  non -negative  numbers. 

1.  No. 

2.  Image  of  right  half  of  real  nianber  line  came  to  mind  to  make  sxire 

we  weren't  dealing  with  negative  numbers. 

MWW>»A»-^ 

0 

3.  The  set  is  infinite  to  the  right. 

SET  13;  The  closed  interval  13,  sj 

1.  Yes  m=5,  no  image 

2.  I  just  saw  two  numbers. 
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3    5 
3.  Knew  answer  as  soon   as  I  saw  It. 

SET  17  ;  {\/2  y3  ,^)/5  y/e  >/7 ,  . . .  j. 

1.  No,  realized  set  was  infinite 

2.  Did  not  look  at  \/~~ 

SET  18 ;  {  Tan  x,  where  x  =T,  TTT,    sT^T^T^T-'-J 

1.  Oh  no  —  not  a  trignometric  function. 

2.  What  does  tangent  mean? 

3.  how  does  it  relate  to  sin  and  cos? 

sin  cos 

tan  =  or  tan  =  — r— 

cos  sm 

4.  I  see  an  infinite  set,  but  may  be  a  trick  question. 

5.  I  will  guess  unbounded. 


SET  21 


:  {  1,  2,    4,  8,  16,  ...}. 


No,  realized  set  was  increasing. 
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S16  K.G. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  nxombers,  R.   An  element, 

m,  of  R  is  said  to  be  eui  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  Had  to  read  the  definition  a  number  of  times  before  I  even  had  the 
vagous  idea  of  what  an  upper  bound  was. 

2.  I  could  not  determine  relationship  between  x  and  m. 

3.  Remembered  to  lose  Venn  diagrams. 


4.   I  now  saw  that  m  had  to  be  i'n  R  and  had  to  be  greater  or  equal  to  x. 


PRACTICE  PROBLEM;   Let  E  be  the  set  ^1,   2,   3,  4^,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  an  upper  bound? 


1.  This  is  not  type  of  problems  I  expected,  although  I  should  have 

from  definition. 

2.  It  seems  obvious. 

3.  I  know  5  or  greater  is  an  upper  bound,  but  is  4? 

4.  Yes. 

SET  2;   The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers L 

1.  Read  problem 

2.  I  believe  that  no  upper  bound  exists  because  the  numbers  go  to  infinity. 

5> 


3.   I  remember  from  a  previous  course  the  conflict  cibout  Q  being  in  the 
natural  numbers. 
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SET  4;   A  finite  set. 

1.  I  cannot  see  an  m  in  the  set. 

2.  I  re-read  the  definition. 

3.  Since  no  specific  values  eire  given  —  how  can  I  determine  an 

upper  bound. 

4.  I  am  confused.   I  need  numbers. 

5.  I  guess  no  upper  bound. 


SET  5; 


£l,  1/2,  1/3,  ...} 


1.  First  noticed  an  infinite  set  and  concluded  no  upper  bound. 

2.  but,  wait,  it  is  decreasing  -»  the  largest  number  mxist  be  the 

upper  bound  =  1. 


SET  6;  The  set  of  real  numbers,  R. 

1.  I  imagine  all  types  of  numbers 

(rational ,  irrational ,  natural ,  etc . 1 

2.  I  see  it  going  on  forever. 

3.  No  upper  bound. 

SET  11;   The  non-negative  numbers. 


1.   Had  trovible  figuring  out  non-negative .   First  thought  numbers  to 
the  left  of  zero. 

^ 1 


0 
2.   Then  thought  of  positive  numbers,  so  no  upper  bound. 
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SET  13:   The  closed  interval 


[3,  sj 


1.   Immediately  thought  upper  bound. 
SET_17:  [\/2V/3,V^>/5,V^>/7,  ...^ 

1.  First  noticed  square  root  sign. 

2.  Then  noticed  progression  with  no  upper  bound. 

3.  Then  asked  if  square  root  would  increase  to  00 

4.  Decided  it  would,  so  no  upper  bound. 

SET  18:  fTan  x,  where  x  =T,  2^  3?^  4T,   ST,   ST,    ...J 

1.  Originally  thought  no  upper  boxmd  because  of  infinite  progression. 

2 .  But  then  realized  that  the  value  for  tan  =  0  at  the  given  x  values , 

3.  There  is  a  upper  bound. 


SET  21 


:  £1,  2,  4,    8,  16,  ...} 


1.  Rapidly  increasing  n\unbers,  so  probably  no  upper  bound. 

2.  No  mental  imagery. 

3.  Saw  a  geometric  series,  but  of  no  importance.   The  set  is  infinite, 
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S17  E.T. 

DEFINITION ;  Let  E  be  a  subset  of  the  real  niombers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  Discovered  what  the  definition  talks  about,  and  what  elements  were 

in  it.  Then  tried  to  relate  the  elements  to  each  other. 

2.  m  must  be  in  R  and  greater  or  equal  to  x,  which  is  in  E. 


E^-  X  —  m  -^R 

PRACTICE  PROBLEM;   Let  E  be  the  set  £  1 ,  2,  3,  4  J,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 

1.  From  the  definition 

E^—  1,  2,  3,  4  -^  m  — ^R 

2.  Therefore  any  number— 4  is  the  upper  bound. 

SET  2 ;  The  set  of  natural  numbers,  N  (i.e.  the  positive  integers!. 

1.   First  drew  mental  image  of  the  set 

Vl,  2,  3,  4,  '''J       and  saw  a  number  line  going  into  infinity- 


2.   I  decided  that  there  is  no  m  which  is  greater  or  equal  to  the 
largest  x,  »*,  no  upper  bound. 


SET  4;   A  finite  set. 

1.      What  is  meant  by  a  finite  set? 

A  specific  nvanber  of  points  ^countable. 

For  example:  \^^,   2,    3,   4  V, 
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2.  Since  the  set  is  coiantable  it  has  an  tipper  limit. 

3.  Remembered  the  term  coxon table  from  discreet  math  or  something. 


SET 


_5:[l,  1/2,  1/3,  ...J 


1.  I  first  tried  to  find  a  pattern,  but  could  not  find  one. 

2 .  Recognized  "...". 

3.  Infinite  set  and  therefore  not  bounded. 

SET  6;  The  set  of  real  numbers,  R. 

1.  I  am  a  little  confused  —  we  are  working  with  the  real  nxambers 

(from  definition) . 

2.  If  a  set  is  a  siobset  of  itself  then  there  is  no  upper  bound. 

image: 


SET  11;   The  non-negative  numbers. 


1.  First  read  "negative  nximbers".   A  line  which  moves  to  the  left 

instead  of  right. 

2.  Read  again  and  saw  line  moving  to  the  left 


3.   No. 


SET  13:  The  closed  interval 


[3,  sj. 


1.   Immediately  related  this  to  problem  before, 
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Closed  interval  -  finite  -  upper  bovmd. 
2 .   Used  no  imagery  except  noticed  f  J  - 

SET  17;  iv^^/sy^ysyiy?,  ...  ^ 

1.  No. 

2.  Again  "..."  was  seen.   Infinite  set  —  no  upper  bo^and. 

SET  18;  f  Tan  x,  where  x  =7^,  2lf,    zT,   ^T,   sT,   ^T,    ...1 

1.  I  always  had  problems  with  tangent  fxonction. 

2.  The  only  images  I  see  is  the  words, 

SIN    COS 
but  don't  know  how  they  relate  to  the  W'  values. 


SET  21 


;  /  1,  2,    4,  8,  16,. . .  1 


1.  Look  at  set. 

2.  Notice  doubling  ^^ some  type  of  geometric  progression, 

3.  No  upper  bound/  no  imagery. 
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S18  J.L. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  I  remember  a  definition  similar  to  this  for  the  upper  bound. 

2 .  X  _y  E  —^  R  and  m  — ^  R 
must  hold  that 

m  31  X 

PRACTICE  PROBLEM;   Let  E  be  the  set-£l,  2,  3,  4  J- ,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.  Does  E  have  an  upper  bound? 

1.  I  visualize  the  definition 

1,  2,  3,  4  — 5>  E  -^  R  CrealsL 
m^  1,  2,  3,  4 

2.  Reals  are  easy  to  define. 

3.  E  was  also  easy  to  define. 

4.  Because  I'd  made  an  equation  for  m,  I  could  plug  the  elements  of 

E  into  X  to  convince  myself  that  4  was  the  upper  bound. 

SET  2 ;  The  set  of  natural  numbers,  N  (i.e.  the  positive  integers).. 
1.  The  numbers  are  positive  and  move  to  right. 


12   3  4 


2.   Nothing  can  be  to  right  of  all  of  them,  therefore  upper  bound  exists, 


SET  4;   A  finite  set. 


1.   Same  as  practice  problem 
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2 .  Yes . 

3.  Thought  about  answer  iininediate. 


SET 


_5_:     {l,    1/2,    1/3,    ...^ 


1.  Another  sequence. 

2.  1,  1/2,  1/3,  ...  — 5r  E  — ^R 

m  ^  1,  1/2,  1/3,  ... 

3.  Descending,  therefore  m  exists. 

4.  Upper  bound  is  anything  equal  to  or  greater  than  1, 

SET  6;   The  set  of  real  numbers,  R. 

1.  Clearly  No! !! 

2.  It  is  just  like  N,  moves  to  the  right. 

no  image 

no  upper  bound 

SET  11;   The  non-negative  numbers. 

1.  The  word  non-negative  confusing. 

2.  Must  be  positive  numbers. 

As  before  move  off  to  the  right. 

3.  No  image. 

No  upper  bound. 


SET  13:   The  closed  interval 


\3.    5] 
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1.  Knew  the  cinswer  before  w-iting. 

2.  Thought  that  whether  the  interval  is.  open  or  closed  is  irrelevant, 

SET  17;  ^v^x/sA/^yiyey?,  ...   1 

1.  Another  sequence. 

2.  Square  root  is  unboionded  r^/n     -   ^^ 

3.  Therefore  no  upper  bound  exist. 

4.  No  image. 

SET  18 ;  £  Tan  x,  where  x  =>7',  if,   ZT,   ^T>    5/^  ^T,    ...  i. 

1.  This  made  me  think  about  high  school  math. 

2 .  Sequence . 

tan  =  y/x  =  0 

3.  Upper  bound  exists. 


SET  21: 


{l,  2,    A,    Q,    16,  ...]. 


1.  Some  type  of  geometric  sequence. 

2.  Obviously  fast  disappearing  to  right. 

3.  No  upper  bound. 
No  image. 
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S19  L.N. 

DEFINITION ;  Let  E  be  a  subset  of  the  real  rnjmbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1. 


2. 


-zTS- 


PRACTICE  PROBLEM;   Let  E  be  the  setXl,    2,   Z,   4 J  ,   i.e.    the  set  consisting 
of  the   first   four  positive  integers.      Does  E  have  cin  upper  bound? 


1.  ,      >      f     r  4 

12      3     4 

Look  to  right  for  highest  order  nvimber. 
SET  2 ;  The  set  of  natural  mambers,  N  Ci.e.  the  positive  integers) 


I ' • *HS  etc. 

I   ^   2    3  ^* 


2.  Asked  myself  if  natural  was  correct  term. 

Drew  line,  checked  to  make  sure  I  had  not  included  zero, 

3.  Where  is  set  1? 


SET  4:   A  finite  set. 


y  If  2,  3,  .  .  . ,  n^ 
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2.  What  does  he  want? 

I  think  of  points  in  line  or  an  array. 

3.  Since  only  specified  number  of  points,  upper  bound  exists  and 

must  be  n. 

4.  Where  is  set  3? 


SET  5 


:  ^  1,  1/2,  1/3,  ...^ 


1.  Ra%i©Rai-»ember9-  —  not  all  there. 

infinite  set 

(Wn=l  to  CO 

Q,  oJ=>o 

2.  Decreasing,  largest  value  is  upper  bound. 

3.  Are  these  questions  or  stimuli? 

Are  they  rauidom?  Is  that  why  the  set  numbers  vary? 

SET  6;  The  set  of  real  numbers,  R. 


1.  4 1 5>   Ee 

solid  line 

usually  think  of  Re  at  end  of  line. 

2.  Infinite,  thus  unbounded. 

SET  11;   The  non-negative  numbers. 


1. 

0,  1,  2,  3,  ....  etc . 

2.   Include  zero  this  time. 
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SET  13;      The   closed  interval   13,    5 J • 

X.     f 3 

3         5 

2.  Open  interval  uses  C   L- 
Closed  interval  uses  T   J 

3.  5 

SET  17;  J  V2  ,sn  y^ ,\/% y^ yr ,  • . .  Z 

1.  Integers  ^^    1 

2.  Why  so  cryptic? 

I  am  looking  for  small  details. 

3.  No  bound   "^  infinity . 

SET  18;  /  Tan  x,  where  x^T,    2f,  zT,   ^T,    sT,   sT,    ...  J" 

1.  What  goes  on  here? 

2.  What  is  he  trying  to  do  with  this  set? 

3.  What  can  he  hope  to  gain  from  this  experiment? 

Looks  like  a  psych,  experiment. 

4.  Why  so  much  bleuik  paper? 

SET  21;    /  1,    2,    4,    8,    16,    ...1 

•"••     t^    J    n=0,l,2,... 
2.      Infinite  set. 
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S20  T.M. 

DEFINITION ;   Let  E  be  a  sxibset  of  the  real  nimbers,  R.   An  element,  m, 
of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 
m  for  every  x  in  E. 

1.  Very  confusing.   I  start  to  panic. 

2.  Relate  subset  E  to  point  m. 


^ 


3.   Look  at  "less  than"  or  "equal  to"  part  of  definition. 

PRACTICE  PROBLEM;  Let  E  be  the  set  |  1,  2,  3,  4 1  ,  i.e.  the  set  consisting 
of  the  first  four  integers.  Does  E  have  an  upper  bound? 

1.   Yes. 

SET  2;  The  set  of  natural  numbers,  N  Ci.e.  the  positive  integers). 

1 .  Savr  this  : 

■  tf   ft   «   • 

2.  Not  paying  attention  to  the  left,  but  infinite  in  both  direction 

3.  Concluded  no. 

3.   "Is  N  bounded?"  No. 

SET  4;   A  finite  set 

1.  I  don't  see  any  nxjmbers. 

2.  What  happens  if  the  finite  set  has  infinite  points 
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3.  How  can  I  assign  an  m  to  this.. 

SET  5;  £  1,  1/2,  1/3,  ...1 

1.   I  see  a  sequence  going  to  infinity 


1 
2.  Therefore  must  be  bounded. 

SET  6;  The  set  of  real  numbers-,  R. 

1.  Automatic  reaction  to  infinite  set,  thus  could  not  define  m. 

2.  No  mental  image. 

3.  Definition  mentions  R  haw  does  it  relate  to  this  set? 

SET  11;  The  non -negative  numbers. 

1.  Half -line  in  mind    ^  positive. 

2.  Here,  I  realized  no  upper  boxand. 


SET  13:  The  closed  interval 


t-  0 


1.  First  see  the  word  interval,  indicates  no  upper  boxmd. 

2.  Infinite  nxmber  of  points. 

'infinite 


^intinite  -i 


3.  No. 
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SET  11 -S^f^  ,\l3  ,\/A  yS  ,\f^  ,\/l ,    ..A 


1.  No  upper  bound. 

2.  I  changed  to  more  conventional  numbers 

1.14,   1.492,  2,      etc. 

3.  Then  I  was  convinced  the  series  ascended  and  was  infinite. 

SET  18;!  Tan  x,  where  x  =^T ,   if,    ^T,   ^Y,    ST,   Sf,    ...1 

1.  No  upper  bound. 

2.  Like  the  previo\is  problem  —  I  am  convinced  the  series  is  ascending 

and  is  infinite. 

3.  No  mental  image. 

SET  21;   /l,  2,    4,  8,  16,  . . .  1 

1.   The  progression  is  ascending  very  rapidly  to  the  right 
^   fast   ^ 


2.   It  is  infinite  ^^  no  upper  bound. 
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S21  A.C. 

DEFINITION ;   Let  E  be  the  sxibset  of  the  real  numbers,  R.   An  element, 
m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 
m  for  every  x  in  E. 


Image : 


2.   I  try  to  relate  the  elements  on  the  line. 

Draw : 

upper  bound 


PRACTICE  PROBLEM:  Let  E  be  the  set£l,  2,  3,  ^j  ,   i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  I  see  4  points  or  elements. 

2.  Hazy,  dark  with  white  circles. 

3 .  Finite . 

4.  Yes. 

SET  2:   The  set  of  natural  numbers,  N  (i.e.  the  positive  integers!. 

1.  Got  side-tracked  on  definition. 

2.  Remember  the  term  "cotintably  infinite". 

The  set  numbers  does  approach  infinity 

3.  I  visualize  myself  counting  the  positive  integers  and  saw  there 
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would  be  no  end  to  the  cotjnting. 
4.  No  upper  boimd. 

SET  4;  A  finite  set. 

1.  This  does  not  seem  to  fulfill  all  the  reqioirements  of  definition, 

2.  But  finite  means  an  ending. 

If  there  is  an  end  then  there  must  be  an  m  greater  than  the  end. 

3.  Its  finite ^#  therefore  upper  bound. 


SET  5; 


^  1,  1/2,  1/3,  ...^ 


1.  The  infinite  series  converges  to  0. 

2.  The  answer  is  the  largest  element  of  the  set. 

m  =  1  or  greater. 

SET  6;  The  set  of  real  numbers,  R. 

1.   I  visualize  a  single  straight  line  with  no  end  in  either  direction. 


2.   I  conclude  there  is  no  bound  in  either  direction. 
SET  11;  The  non-negative  numbers. 

1.  What  are  non-negative  numbers? 

2.  Answer:  positive  numbers. 

3.  Positive  numbers  go  to  infinity. 

4.  Therefore  no  bound  Cupper X. 

5.  No  image. 
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SET  13;  The  closed  interval  3,  5 

1.  Closed  interval  is  finite. 

2.  Is  it? 

Dravr: 

3.  No,   interval  is  not  finite. 

But  closed  means  endpoint. 

4 .  Yes ,  m=5 

SET  17;^  vT^Vf^vI^VS^v/^.w^,  ...I 

1.  Tried  to  determine  if  v  of  numbers  was  increasing  or  decreasing. 

2 .  Saw  the  dots . 

3.  I  better  determine  if  increasing  or  decreasing. 

4.  Increasing. 

5.  The  set  is  infinitely  large. 

6.  No  upper  bound. 

SET  18;^  Tan  x  where  x  =T ,   2T,   ^f,   4/^,  sf,   ^T,    •••  J' 

1.  Tried  to  determine  if  tan  x  is  increasing  or  decreasing. 

sin  X 

2.  I  remember:   tan  x  =  

cos  X 

But  can't  relate  the  ff    values  to  them. 

3.  Saw  the  dots. 
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4.  The  set  is  increasing. 

5.  The  set  is  infitiitely  large. 

6.  No  upper  bound. 


SET  21: 


^1,  2,    A,    8,  16,  ...y 


1.  As  before,  the  set  increases  rapidly. 

2.  Saw  the  dots. 

3.  No  upper  bound. 

4.  No  image,  because  obvious:. 
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S22  J.T. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 
m,  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 
m  for  every  x  in  E. 

1.  e[r] 

2.  E  =  fx.  is  an  element  I 

3.  m  ^   X   —^  R 

4.  The  meaning  of  upper  bound  is  ambiguous.   I  hope  to  figure  it  out 
after  doing  some  problems-. 

PRACTICE  PROBLEMS;   Let  E  be  the  set i  1 ,  2,    3,    4 J,  i.e.  the  set  consisting 
of  the  first  four  positive  integers.   Does  E  have  an  upper  bound? 


1.  E  =  1,  2,  3,  4 

2 .  1,2,3,4  are  positive . 

3.  Read  definition. 

4.  where  does  m  fit? 

5.  m  is  a  real,  so  is  x. 

6.  Aha,  m— X,  therefore  4  or  greater. 

7.  Let's  try  next  problem. 

SET  2 ;  The  set  of  natural  numbers,  N  (i.e.  the  positive  integers).. 

1.  E  ={  1,  2,  3,  4,  ...y 

image:   Think  of  positive  integers  and  natviral  numbers. 

2.  Read  definition. 

3.  The  set  goes  to  infinity.   Does  that  mean  if  m  =  ^^it  is  the  upper 

bound? 
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4.  Infinity  is  not  a  real  niimber  ,*^  no  upper  bound. 

5.  I  am  still  confused  with  definition. 

SET  4;  A  finite  set. 

1.  This  is  the  same  as  the  practice  problem. 

2.  Yes,  upper  bound. 

3.  I  didn't  use  definition  or  pictures  for  this  set. 


SETS 


:  ^  1,  1/2,  1/3,    ...J. 


1.  This  set  is  also  like  the  precious  problem. 

2.  Turn  to  definition. 

3.  Confused. 

4 .  Draw . 


5.   It  is  infinite  and  therefore  there  must  be  no  m,  no  upper  bound. 
SET  6;  The  set  of  real  numbers,  R. 


1.   Infinite  series  in  both  direction,  not  just  to  the  right.   Zero 
as  center. 


2 .    image :       ^ , 

^ 1 


3.   Getting  the  hang  of  definition. 

If  no  m  or  infinite  no  upper  bound. 

SET  11;   The  non-negative  numbers. 
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1.  Infinite  series  to  the  left  Cno  imaged. 

2.  Same  as  previous  problem. 

No  m  euid  infinite  means  no  upper  boiind, 

3.  The  problems  are  going  quicker  novr. 


SET  13;   The  closed  interval 

1.  This  is  finite. 

2.  Upper  bound. 


[3.=] 


SET_17:Jn/2,\/3,V/4,V/5,V/6,\/7,  ...^ 

1.  Infinite  series. 

images :      /— 

V 

2.  numbers  seem  to  be  i'ncreasing. 

3.  As  before,  no  m,  no  upper  bound. 

SET  18;f  Tan  x,  where  x  =T,  "^.J,    if,   ^T,    ^f,   ^T,    •  • -J- 

1.      Trig  fxinction  are  cyclic 
Image : 


G2^ 


2. 


^y 


3.   Since  infinite  sequence,  graph  of  tan  fiinction  goes  to  infinity 
somewhere . 
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4.   No  upper  bound. 

.:  ^1,  2,    A,    8  ,16,  ...^ 


SET  21; 


1.   Saw  familiar  pattern,  2 


2.  Saw  dots. 

3.  Quickly  decided  it  was  infinite, 

4.  No  upper  bound. 

5.  No  imagery. 
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S23  J.B. 

DEFINITION ;  Let  E  be  a  subset  of  the  real  niombers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bovmd  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  First  I  have  to  think  what  the  term  stibset  means. 

2.  I  think  about  a  set,  which  is^  a  collection  of  things: 

(i.e.  red,  blue,  green 1 

3.  A  Venn  diagram  comes  to  mind,  using  real  numbers  as  the  universe. 


A  subset,  must  therefore  be  a  small  set, 
within  R. 


Looking  at  the  Venn  diagram,  I  think  about  the  upper  bo\and  as 
being  everything  in  the  universe  R  with  the  exception  of  the 
area  within  the  small  circle. 


Let  E  be  the  set  Cl,  2,  3,  4^  ,  i.e.  the  set  consisting 


PRACTICE  PROBLEM; 

of  the  first  four  positive  integers. '  Does  E  have  an  upper  bound? 


1.  This  subset  contains  a  finite  set  of  nijmbers. 

2,  They  are  increasing  in  size. 


12  3  4 
■4 • • — ^ 


146 


S23   J.B. 
3.      Since  it  is   finite  the  largest  niomber  is   4.      The  upper  bovind  =  4, 
SET  2 ;      The  set  of  natiiral  numbers,   N    Ci.e.    the  positive  integers). 

1.  First  thing  that  comes  to  mind  is     —  what  is  a  natural  nvunber. 

2.  The   series  is  increasing 

0     12      3,^ 


3.  Its  an  infinite  series  going  to  infinity  (  ^^  \ 

4.  No  upper  bound. 

SET  4;   A  finite  set 

1.  what  is  a  finite  set? 

2.  Its  a  specified  nxomber  of  things. 

■«   •  « m  a  a — ^ 

or 

fa,  bj,[j:  ,^ 

3.  Therefore  it  has  an  upper  bound. 


SET  5; 


^1,  1/2,  1/3,  ...]. 


1.  Look  at  the  dots  indicating  a  infinite  set. 

2.  Infinite  sets  do  not  have  upper  bovinds. 

3.  Looking  at  the  numbers  they  are  decreasing  going  towards  0. 

^  1/3    1/2       I 

4.  Upper  bound  of  1. 
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SET  6;   The  set  of  real  nvimbers,  R. 

1.  What  is  the  set  of  real  numbers. 

2.  Remember  the  Venn  diagram  r  used  in  the  definition. 


3.   No  upper  bound,  since  it  goes  to  infinity. 


SET  11:    _  ^..      . 
The  non-negative  numbers. 


1.   This  brings  back  set  2  (natural  #  -  positive  integers) 


oo 


2.   It  goes  to  infinity,   therefore  no  upper  bound. 
SET  13;   The  closed  interval  l3,  51 


1.   Immediate  reaction  of  upper  bound  of  5,  since  it  is  a  finite 
set. 

3       5 

9  0 


SET  17;  ^v/2  ,v5,v/4  ,\/S  ,y/i ,y/l ,    ...  I 


1.   First  looked  at  dots,  indicating  an   infinite  set,  therefore 
no  upper  bound. 

i.e.     n/1  =  2 

\/9  =   3 

3.   Indicates  the  larger  the  number  gets  —  the  larger  the 
square  of  it. 
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4.   No  upper  bound  goes  to  oo 

SET  18;   Tan  x,  where  x=   ,2,3,4,5,6, 

1.  Had  to  remember  what  tangent  is. 

2.  Drew  picture  to  figure  it  out. 


cos 


sine 


sm 

tan  = 

cos 


3.  What  where  the  values  of  sin  &  cos  at  //  ,   2  "JlT,   etc. 


COS 


4.   tcui 


sm 
cos 


..    =  0 


tan  2 


sin  2 
cos  2 


=  a 


5.  Therefore  the  set  is   0,  0,  0,  ... 

6.  Eventhough  infinite  set  the  numbers  are  constant  0. 

upper  bound  of  0. 


Therefore 


SET  21 


:  £l,  2  4,  8,  16,  ...7 


1.  Its  an  infinite  series  —  does  it  have  an  upper  bound. 

^  T,   ^  •      -  Cn-1) 

2.  Equation  is  2 

3.  When  n  becomes  so  large  the  -1  will  have  no  effect. 

4.  Increasing  set  and  since  n  can  go  to  infinity  —  no  upper  bound. 
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2.   This  is  true  for  all  finite  sets. 


SET  5; 


^1,  1/2,  1/3,  ...J 


1.  Saw  the  form  1/n. 

2.  Looked  at  consecutive  pairs;   saw  that  denominator  was  increasing. 

3.  Infinite  set  and  therefore  upper  bound. 

(no  image  I 

SET  6;  The  set  of  real  numbers,  R. 

1.  From  memory  R  has  no  upper  bound. 
I  don't  think  I  used  the  image. 

2.  I  hesitate  to  use  too  many-  images  because  they  are  xinreliable. 

SET  11;  The  non-negative  numbers. 

1.  I  think  what  it  means  to  be  non-negative ,  getting  an  image 
of  —  0.  Again  I  see  an  infinite  set,  and  know  that  it  is 
unboxmded . 

2.  I  note  that  it  is  bounded  below. 


SET 


13;   The  closed  interval  |3,  5j 


1.  An  interval  means  it  is  infinite,  but  also  it  is  bracketed  on 

both  sides 

L    '^    ] 

2.  In  this  problem  the  infinite  set  is  bounded  by  3  and  5,  therefore 

it  has  an  upper  bound. 
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S24  S.M. 

DEFINITION ;   Let  E  be  a  subset  of  the  real  numbers,  R.   An  element, 

m,  of  R  is  said  to  be  an  upper  bound  of  E  if  x  is  equal  to  or  less  than 

m  for  every  x  in  E. 


1.  m^  R 

2.  X  ^  m 

3.  The  above  equation  is  what  I  thought  about  when  reading  the 

definition. 


Let  E  be  the  set-fl,    2,  3,  4  ?  ,  i.e.  the  set  consisting 


PRACTICE  PROBLEM; 

of  the  first  four  positive  integers."  Does  E  have  an  upper  bound? 

1.  4  elements  in  the  set  (finite  set) 

2.  Therefore  there  is  an  upper  boiond. 


SET  2;   The  set  of  natural  nijmbers,  N  (i.e.  the  positive  integers). 

1.  I  focus  on  a  particular  set  of  numbers. 

12  3  4  ...  etc 

2.  I  can't  tell  the  last  element  of  E,  since  the  number  go  to  infinity, 

3.  I  wonder  if  these  do  have  upper  bound? 

4.  I  can't  see  an  end  to  this  set  -  so  no  upper  bound. 

SET  4;  A  finite  set. 

1.   Concrete  at  end  and  beginning. 

Draw: 

S  a,  b,  c,  d,  e,  f ,  g  I 

If  I  use  above  set  where  a   b   c   d   e   f   g,  then 

g  is  the  upper  bound. 
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3.  Confused,  what  woiild  be  the  answer  if  the  question  was; 

The  closed  interval      i-  oo  ,  +  oq  j 

4.  My  answer  for  this  problem  is  upper  bound. 

SET  17;  5  \/2,  i/^yT y 5  ,\/6  yT,  . . .  Z 

No,  realized  set  was  infinite. 

1 .  Ignored  V 

2.  Saw  dots. 

SET  18 :  i  Tan  x,  where  x  =  ^,  2%  iT,   47J  ^7,   &T,    ... 
1.   I  tried  to  visualize  graph  of  tan  x: 


2.  Disregarded  graph,  because  it  didn't  tell  me  what  the  values 

were  at  -^  ,  2^,  etc. 

3.  Went  to  mathematical  form: 

sin 

tan  = 

cos 

4.  Not  sure  what  the  values  are  tox'lT   cuid  27f 

5.  See  dots. 

6.  The  set  is  infinite,^  no  upper  bound. 

7.  I  did  not  really  solve  the  tan  problem. 


SET  21: 


£  1,  2,  4,  8,  16,  ...  j 


•1.   No,  realized  an  infinite  geometric  increasing  set. 
no  image  (automatic) 
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